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Abstract 


Allocating  resources  is  never  an  easy  task,  especially  when  requirements  call  for 
more  resources  than  those  available.  It  gets  more  difficult  when  the  availability  of 
resources  shrinks  from  year  to  year.  This  is  the  case  at  the  National  Air  Intelligence 
Center  (NAIC)  located  primarily  at  Wright-Patterson  Air  Force  Base  in  Ohio.  In  each  of 
the  past  five  years,  NAIC  has  been  forced  to  make  their  resource  allocation  decision  with 
fewer  dollars  and  less  manpower.  This  decision  has  been  time  consuming,  manpower 
intensive,  and  sometimes  very  heated.  In  an  effort  to  lessen  these  three  consequences,  a 
resource  allocation  model,  based  upon  the  NAIC  Commander’s  values  and  preferences, 
was  developed.  The  methodology  for  the  model  is  founded  upon  decision  analysis  with 
value-focused  thinking.  A  scoring  sheet  based  on  measures  which  were  taken  from  the 
commander’s  values  was  also  developed.  Using  multi-attribute  utility  theory,  the 
measures  were  scored  using  scoring  functions  and  then  multiplied  by  the  commander’s 
preferences  to  determine  an  overall  utility  score.  For  the  FY99  budget  cycle  at  NAIC,  62 
unique  funding  proposals  were  scored  and  ranked  using  28  measures  for  each  proposal. 
The  developed  value  hierarchy  allows  NAIC  to  choose  their  own  alternatives  based  on  this 
ranked  list.  Significant  differences  in  perceived  impact  exist  between  recommended 
proposal  cuts  and  the  list  of  proposals  approved  for  cuts  by  NAIC.  Some  sensitivity 
analysis  was  performed  on  the  commander’s  preferences. 


IX 


I.  Introduction 


1.1  Background 

Since  1989,  the  Air  Force's  budget  and  manpower  levels  have  decreased  while 
operations  tempo  has  increased,  requiring  units  to  do  more  with  less  [1;  13;  34].  As  such, 
more  demands  are  placed  on  the  individual  units  each  year  [36].  These  demands  require 
units  to  continually  look  for  new  and  innovative  means  to  manage  and  allocate  their 
declining  resources.  The  National  Air  Intelligence  Center  (NAIC)  is  one  such  unit.  Their 
Decision  Unit  (DU)  process  for  allocating  resources  is  creative,  but  it  is  time  consuming 
and  makes  picking  the  best  alternative  difficult  [39].  The  programs  or  projects  designated 
to  be  trimmed  were  typically  selected  with  a  great  deal  of  subjectivity. 


1.2  Problem  Statement 

To  perform  resource  allocation,  NAIC  created  the  DU  process.  This  DU  process 
cuts  across  the  typical  functional  unit  or  line  unit  areas  of  NAIC  [23;  28].  This  process 
provides  a  means  to  track  and  allocate  all  of  NAIC's  resources,  including  manpower  and 
budget,  which  are  split  into  the  areas  of  research,  development,  test  and  evaluation 
(RDT&E);  procurement;  external  assistance;  and  operations  and  maintenance  (O&M). 
Within  NAIC’s  DU  process  for  resource  allocation,  many  difficult  decisions  are  made. 
Currently,  the  NAIC  Commander  is  presented  a  set  of  alternatives  where  each  alternative 
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is  a  percentage  of  NAIC’s  resources.  The  commander  must  select  a  certain  number  of 
alternatives  or  provide  his  own  in  order  to  determine  the  resource  allocation  strategy  for 
that  year.  He  uses  his  experience,  his  value  system,  and  the  Senior  Management  Team's 
(SMT)  opinion  to  make  this  decision.  However,  he  has  no  tools  to  help  quantify  the  value 
of  the  different  choices  he  faces. 


1.3  Research  Objective 

The  intent  of  this  research  is  to  develop  a  methodology  that  would  allow  the  NAIC 
Commander  to  choose  alternatives  for  NAIC’s  allocation  of  resources  consistent  with  and 
based  upon  his  values  and  preferences  that  show  his  objectives  for  NAIC.  A  secondary 
goal  is  to  develop  a  method  that  would  significantly  shorten  the  length  of  time  it  takes  to 
generate  equal  or  better  alternatives  than  those  currently  being  presented  to  the 
Commander.  This  thesis  explores  techiuques  to  rank  alternatives  and  processes  to 
generate  alternatives  in  order  to  help  the  NAIC  Commander  make  a  more  timely  and  value 
based  decision  on  his  unit's  resource  allocation  strategy. 


1.4  Scope 

The  research's  intention  is  not  to  alter  or  redesign  NAIC's  DU  process.  Instead,  it 
seeks  to  assist  the  commander  by  providing  good  alternatives  and  quicker  and  more 
consistent  evaluation  of  alternatives  within  the  DU  process.  Resource  allocation 
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alternative  generation  is  a  major  bottleneck  in  the  DU  process  requiring  many  iterations 
and  much  time.  This  research  examines  methods  that  help  evaluate  and  rank  alternatives 
and  methods  that  provide  for  alternative  generation.  The  expected  outcome  is  a  decrease 
in  the  time  it  takes  to  provide  resource  allocation  alternatives  based  on  the  decision 
maker's  values  and  preferences,  less  subjectivity  and  less  political  maneuvering. 


1.5  Overview  of  Thesis 

Chapter  n  provides  a  brief  literature  review.  Specifically,  it  provides  an  overview 
of  the  DU  process,  explores  other  resource  allocation  techniques,  examines  decision 
analysis  and  value-focused  thinking,  and  provides  a  brief  overview  of  the  software 
packages  DPL  and  Logical  Decisions.  The  Decision- Analysis  Process  Flowchart  is  also 
presented  in  this  Chapter  along  with  a  short  examination  of  the  knapsack  problem. 

Chapter  El  is  based  upon  the  Decision-Analysis  Process  Flowchart  and  addresses 
the  first  three  blocks  in  that  flowchart  to  include  identifying  the  decision  situation  and 
understand  objectives,  identify  alternatives,  and  decompose  and  model  the  problem. 

Chapter  IV  documents  the  changes  that  occurred  for  FY99's  resource  allocation 
strategy.  While  the  software  model  built  in  Chapter  III  does  not  apply,  the  methodology 
that  was  used,  decision  analysis  with  value-focused  thinking,  still  applies  and  was  used  to 
score  and  rank  the  proposals.  This  chapter  addresses  the  next  two  blocks  in  the 
flowchart:  choose  the  best  alternative  and  perform  sensitivity  analysis.  In  other  words,  it 
presents  and  discusses  the  results  of  the  methodology  used  for  NAIC.  Also,  this  chapter 
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compares  the  recommendations  arrived  at  through  this  methodology  with  NAIC’s  current 
resource  allocation  strategy. 

Chapter  V  deals  with  the  final  two  blocks  in  the  flowchart  by  addressing  whether 
or  not  further  analysis  is  needed.  The  chosen  alternative  was  not  intended  to  be 
implemented,  only  the  methodology  examined  for  inelusion  into  NAIC’s  resouree 
allocation  process  in  years  to  come.  Conclusions  are  made  and  other  insights  gathered 
during  the  analysis  are  presented. 
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11.  Literature  Review  and  Methodology 


2.1  Overview 

While  this  research  seeks  to  generate  and  rank  alternatives  and  to  assist  in  the 
selection  decision  for  NAIC's  resource  allocation  strategy,  a  thorough  understanding  of 
NAIC's  DU  process  is  required.  Since  it  does  not  follow  the  traditional,  functional  means 
of  allocating  resources,  this  process  is  difficult  to  understand,  but  it  is  currently  the 
method  that  NAIC  uses  to  provide  its  resource  allocation  alternatives  to  the  commander 
[28].  Knowing  the  DU  process  also  provides  a  better  understanding  and  background  for 
the  problem  statement.  After  the  DU  process  is  discussed,  other  resource  allocation 
techniques  are  explored.  Decision  analysis  and  value-focused  thinking  are  examined.  The 
software  that  was  used  to  model  this  problem  is  reviewed  and  explained.  Lastly,  a  brief 
explanation  of  knapsack  problems  is  provided. 


2.2  DU  Process 

Typically,  the  NAIC  Commander  annually  issues  policy,  guidance  and  baseline 
resources  to  begin  the  DU  process  [25;  26].  From  this  policy,  guidance,  and  baseline, 
each  of  the  19  DU  Managers  prepares  a  DU  plan  based  on  expected  customer 
requirements.  Currently,  each  DU  Manager  includes  four  levels  in  their  plans,  based  on 
incremental  cuts  of  5%.  In  other  words,  the  first  plan  outlines  how  the  DU  would  cut 
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resources  by  5%.  The  other  three  levels  deal  with  resource  reductions  of  10%,  15%,  and 
20%,  respectively.  The  19  DUs  are  broken  down  into  Production  (10  DUs),  Data 
Exploitation  (five  DUs),  and  Infrastructure  (four  DUs).  A  different  scoring  sheet  was 
used  to  score  the  impacts  for  each  of  the  three  areas.  The  DU  Managers  use  these  scoring 
sheets  to  score  their  levels.  These  plans  and  scoring  sheets  are  submitted  to  the  DU 
Council. 

Within  the  DU  Council,  there  is  a  DU  Focus  Group  whose  purpose  is  to  resolve 
any  conflicts  and  perform  any  necessary  research  or  quality  checks  on  the  DU  plans.  For 
example,  a  conflict  in  resource  allocation  might  arise  if  two  or  more  of  the  plans  submitted 
by  different  DUs  call  for  more  than  100%  of  the  available  resources.  After  the  DU  Focus 
Group  fixes  any  discrepancies  it  discovers,  the  results  and  recommendations  are  then 
presented  to  the  entire  DU  Council.  At  this  time,  a  ranked  list  is  developed  based  on  the 
scoring  sheets  and  recommendations  from  the  DU  Focus  Group.  This  ranked  list  is  then 
forwarded  to  the  Senior  Management  Team  (SMT)  which  is  composed  primarily  of  the 
senior  officers  and  civilians  within  NAIC.  The  SMT  reviews  the  ranked  list  and  develops 
a  resource  allocation  alternative  from  that  list  and  presents  it  to  the  commander. 
Considering  the  advice  of  the  SMT,  the  commander  decides  NAIC's  resource  allocation 
strategy  or  determines  that  another  alternative  must  be  designated  or  developed  [27;  28]. 


2.3  Resource  Allocation  Strategies 

There  are  many  resource  allocation  methods  and  strategies  available,  and  this 
section  highlights  just  a  few.  Hardly  a  day  goes  by  without  some  company  being  required 
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to  take  some  type  of  resource  allocation  cut.  Just  a  few  examples  are  General  Motors 
Corporation,  Amtrak,  NASA,  Ford  Motor  Company,  Polaroid  Corporation,  TJX  which 
owns  Marshalls,  Proctor  and  Gamble,  Budget  Rent  A  Car,  and  even  the  IRS  [16;  19;  20; 
31;  32;  33;  35;  38;  41].  While  not  much  information  is  presented  in  these  articles  on  the 
strategy  used  to  accomplish  these  cutbacks,  it  appears  that  much  thought  went  into 
choosing  between  competing  alternatives.  These  alternatives  came  about  through  various 
techniques,  many  of  which  are  documented  in  Chapter  6  of  Clemen’s  textbook  Making 
Hard  Decisions  [8;  187-209].  Probably  one  of  the  easiest,  least  time  consuming  methods 
is  referred  to  as  a  peanut  butter  spread  by  Air  Force  Materiel  Command  (AFMC)/XPM 
[4: 28;  37: 15].  This  strategy  cuts  an  equal  percentage  from  all  organizations.  An  attempt 
was  made  to  develop  a  value- based  model  for  AFMC’s  manpower  reductions,  but  the 
values  and  measures  used  in  this  model  were  difficult  to  obtain  [37: 17-22].  Another 
thesis  effort  was  conducted  on  AFMC’s  manpower  reductions.  This  effort  used  linear 
programming  techniques  in  an  effort  to  generate  manpower  reduction  alternatives  [4:  5]. 
Linear  programming  is  a  very  powerful  technique,  but  for  AFMC,  it  did  not  prove  fruitful. 
Performing  a  cost-benefit  analysis  is  another  method  that  can  be  used  to  evaluate  resource 
allocation  alternatives.  Major  Donald  Barton  used  this  technique  to  help  Unit  Training 
Managers  make  better  decisions  on  their  resource  allocation  alternatives  [2]. 

Charles  Mendoza  wrote  about  resource  allocation  in  the  constmction  industry 
[21].  He  mentions  that  heuristics  and  optimization  techniques,  mathematical  programming 
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and  enumeration,  can  be  used  to  solve  constrained  resource  allocation  problems. 

“Heuristic  approaches  employ  rules  that  have  been  found  to  work  reasonably  well 
in  similar  situations.  They  seek  better  solutions.  Optimization  approaches  seek  the 
best  solutions  but  are  far  more  limited  in  their  ability  to  handle  complex  situations 
and  large  problems”  [21:  44]. 

A  thesis  effort  produced  by  Robert  Buffum  as  a  student  at  the  Naval  Postgraduate  School 
also  documents  different  resource  allocation  strategies  [5].  He  highlights  the  Naval  Air 
Test  Center’s  (NATC)  resource  allocation  strategy  and  then  discusses  two  alternate 
strategies.  To  begin,  he  suggests  that  NATC  uses  an  “incremental  process”  which  focuses 
only  on  the  current  strategy  and  increments  that  strategy  to  varying  degrees.  The 
incremental  process  also  considers  only  a  small  number  of  the  possible  alternatives  for 
their  resource  allocation.  Buffum  points  out  many  of  the  shortcomings  of  that  approach 
[5:  17,18].  He  then  writes  about  a  satisficing  resource  allocation  model  and  an  optimal 
resource  allocation  model.  The  satisficing  model  may  not  provide  all  of  the  necessary 
ingredients  to  obtain  an  optimal  solution  to  the  problem  but  can  be  used  to  show 
management  that  a  better  solution  is  possible.  The  Defense  Logistics  Agency  (DLA)  used 
a  forecasting  model  to  help  allocate  their  resources.  They  said  “a  forecasting  model  is 
required  to  help  managers  identify  the  magnitude  of  the  changes  in  advance  so  that  they 
can  take  proactive  steps  to  match  resources  with  customer  requirements”  [9:  v]. 

Another  method  used  to  help  decide  a  resource  allocation  strategy  is  simulation. 
“The  United  States  Coast  Guard’s  law  enforcement  mission  is  conducted  in  an  atmosphere 
of  uncertainty”  [6:  1].  Simulation  offers  easier  measurement  of  performance,  modeling  of 
uncertainty,  and  statistical  inference.  There  are  many  other  methods  and  strategies  in 
addition  to  the  few  already  mentioned  that  select  a  resource  allocation  alternative.  For 
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NAIC’s  resource  allocation  strategy,  decision  analysis  using  value-focused  thinking  was 
chosen  as  the  method  to  select  the  best  alternative  from  those  generated. 


2.4  Decision  Analysis 

Clemen  says  that  there  are  four  areas  of  difficulty  faced  in  most  decisions  where 
decision  analysis  can  be  helpful.  These  four  areas  are  complexity,  uncertainty,  multiple 
objectives,  and  differing  perspectives  leading  to  different  results  [8:  2-3].  NAIC's  problem 
encompasses  complexity,  multiple  objectives  and  differing  perspectives  leading  to  different 
results.  After  all,  there  are  roughly  19  trillion  possible  resource  allocation  alternatives  to 
consider  for  19  DUs  at  five  levels  each,  where  the  five  levels  are  0%,  5%,  10%,  15%,  and 
20%.  Also,  with  all  of  the  DU  Manager's  scoring  their  percentage  cuts,  there  is  inevitably 
going  to  be  a  certain  amount  of  subjective  judgment.  The  American  Heritage  Dictionary 
defines  subjective  as  "particular  to  a  given  individual;  personal."  According  to  Clemen, 
"The  decision-analysis  approach  allows  the  inclusion  of  subjective  judgments.  In  fact, 
decision  analysis  requires  personal  judgments;  they  are  important  ingredients  for  making 
good  decisions"  [8:  5].  Under  these  guidelines,  NAIC’s  resource  allocation  problem  can 
be  subjected  to  a  decision  analysis  approach.  Figure  1  shows  a  flowchart  for  decision 
analysis  taken  from  Clemen’s  book  which  forms  the  basis  for  Chapters  HI,  IV,  and  V  [8: 

6]. 
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Figure  1.  Decision-Analysis  Process  Flowchart 


2.5  Value-Focused  Thinking 

The  Decision-Analysis  Process  Flowchart  is  followed  in  Chapter  in.  However,  the 
means  for  accomplishing  many  of  the  key  elements  in  that  process  is  value-focused 
thinking.  Ralph  Keeney  developed  the  idea  of  value-focused  thinking  by  suggesting  that 
“values  should  play  a  more  central  role  in  formalizing  decision-making  processes"  and  that 
"value-focused  thinking  should  lead  to  better  alternatives  than  those  generated  by 
‘conventional’  procedures”  [3:  465].  “Value  focused  thinking  essentially  consists  of  two 
activities:  first  deciding  what  you  want  and  then  figuring  out  how  to  get  it”  [15:4]. 

Keeney  distinguishes  between  alternative-focused  thinking  and  value-focused 
thinking.  Alternative-focused  thinking  is  more  or  less  choosing  from  the  obvious 
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alternatives  at  hand  [15:6].  These  alternatives  come  from  some  prespecified  hst,  and 
often  people  are  left  wondering  how  that  list  was  formed  [15:8].  The  decision  maker  just 
has  to  choose  the  best  alternative  from  the  list.  On  the  other  hand,  value-focused  thinking 
begins  with  values;  “Values  are  principles  used  to  evaluate  the  actual  or  potential 
consequences  of  action  and  inaction,  of  proposed  alternatives,  and  of  decisions”  [15:  6,7]. 
Values  are  used  to  form  alternatives.  Why  consider  value-focused  thinking  over  another 
method?  Keeney  emphatically  answers  this  by  saying  that  “value-focused  thinking  should 
lead  both  to  more  appealing  decision  problems  and  to  choices  among  better  alternatives 
than  those  generated  by  happenstance  or  conventional  approaches”  [15:8].  Figure  2  gives 
a  great  overview  of  what  value-focused  thinking  entails  where  the  arrows  represent 
influences  [3: 467]. 

For  NAIC's  problem,  most  of  the  areas  contained  within  Figure  2,  which  shows 
what  value-focused  thinking  entails,  are  accomplished  and  presented  in  Chapters  in  and 
IV.  Figure  2  is  applied  to  NAIC’s  resource  allocation  strategy.  Specifically,  NAIC's 
objectives  are  defined.  A  value  hierarchy  is  formed  where  a  value  hierarchy  contains  the 
values  and  measures  that  define  the  objectives  for  NAIC  in  greater  detail.  Scoring 
functions,  similar  to  utility  functions,  are  developed  to  score  the  measures.  Value-focused 
thinking  helped  create  alternative,  allowed  for  a  large  amount  of  conomunication  with  both 
the  NAIC  Commander  and  the  DU  Managers,  and  provided  some  very  useful  information. 
Data  is  collected  from  scoring  sheets  where  the  DU  Managers  rate  their  percentage  cuts 
and  data  is  collected  on  how  important  each  measure  and  objective  is  to  the  commander. 
Finally,  alternatives  are  evaluated  using  decision  analysis  software.  Two  types  of  decision 
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analysis  software  are  DPL  and  Logical  Decisions  which  are  examined  in  the  following  two 
sections. 


Figure  2.  Overview  of  Value-Focused  Thinking 


2.6  DPL 

DPL  allows  decision  analysis  problems  to  be  represented  in  three  possible  views 
[10:  41].  All  three  are  very  useful  in  conveying  information  to  the  decision  maker  whether 
it  is  by  using  the  decision  tree,  influence  diagram,  or  a  combination  of  both.  The  decision 
tree  term  is  appropriate  because  it  is  similar  to  a  tree.  There  is  one  beginning  structure, 
either  a  decision  node  or  chance  node,  similar  to  the  trunk  of  a  tree.  A  decision  node  is  a 
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rectangle  and  represents  the  possible  alternatives  for  a  certain  decision  and  a  chance  node 
is  an  oval  and  incorporates  uncertainty.  A  value  node  in  the  shape  of  a  rectangle  with 
rounded  comers  contains  a  single  value  in  DPL.  From  the  starting  point,  there  can  be 
many  branches  off  the  tmnk.  In  DPL,  these  branches  can  be  value,  decision,  or  chance 
nodes  springing  from  the  beginning  node.  There  are  also  branches  springing  from 
branches.  When  climbing  a  tree,  an  objective  may  be  to  reach  the  very  top,  to  go  as  far 
out  on  a  particular  limb,  or  just  to  reach  that  bird  nest.  In  DPL,  different  objectives  can 
also  be  designated  and  solved.  DPL  automatically  designates  the  objective  function  as 
attribute  1  if  there  is  only  one  measure.  If  there  is  more  than  one  measure  that  defines  the 
objective  function,  DPL  has  the  capability  to  define  and  track  up  to  64  different  attributes 
or  variables  that  form  the  objective  function.  Attributes  are  also  used  in  DPL  to  define 
constraints  in  the  decision  tree  view.  Constraints  can  be  used  to  prune  the  tree  in  order  to 
lessen  the  number  of  alternatives  considered  and  evaluated.  Figure  3  provides  an  example 
of  a  decision  tree  in  DPL  where  the  path  that  provides  optimality  is  highlighted  in  black. 
The  numbers  under  each  branch  are  the  attribute  values. 

DPL  also  provides  the  capability  to  create  influence  diagrams.  With  many 
decisions  and/or  chance  nodes  and  many  alternatives  within  these  nodes,  the  decision  tree 
can  become  enormous,  ff  Figure  3  were  fully  expanded,  it  would  show  125  different 
branches  or  alternatives.  Key  dependencies,  decisions,  and  uncertainty  also  become 
difficult  to  show  in  a  large  decision  tree.  When  decision  trees  become  too  large,  influence 
diagrams  help  the  decision  maker  and  provide  "an  unambiguous  representation  of 
probabilistic  and  value  dependencies"  [10:  44].  DPL  shows  these  dependencies 
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Figure  3.  Example  Decision  Tree  in  DPL 


through  different  colored  arcs  in  the  influence  diagram.  However,  influence  diagrams  do 
not  show  the  alternatives  within  a  decision  node  nor  the  possible  outcomes  within  a 
chance  node. 

Given  the  strengths  and  weaknesses  of  both  influence  diagrams  and  decision  trees, 
it  is  sometimes  necessary  to  do  both.  Modeling  of  NAIC's  problem  makes  use  of  both  the 
influence  diagram  and  decision  tree. 


2.7  Logical  Decisions 


Logical  Decisions  for  Windows  (LDW)  is  a  software  package  that  ranks 
alternatives,  develops  a  value  hierarchy  with  goals  and  measures,  applies  weights  to  goals 


and  measures,  and  develops  utility  functions  for  the  measures  [17:  14].  LDW  is  used  in 
considerable  detail  to  provide  a  value  hierarchy  for  NAIC,  scoring  functions  for  the 
measures,  and  sensitivity  analysis  on  the  commander's  weights.  While  DPL  has  the 
capabihty  to  perform  sensitivity  analysis,  LDW  provided  a  better  capability.  Specifically, 
sensitivity  analysis  was  used  for  one  weight  at  a  time  while  keeping  the  other  weights 
proportional  and  summing  to  one.  This  was  best  accomphshed  in  LDW. 


2.8  Knapsack  Problems 

Any  integer  program  that  has  only  one  constraint  and  all  of  the  variables  are 
confined  to  being  zero  or  one  is  a  knapsack  problem  [40:  468].  The  idea  behind  this  is 
that  a  knapsack  can  only  hold  so  much.  Not  only  that,  but  it  can  only  hold  an  integer 
number  of  items.  Therefore,  the  most  beneficial  item  goes  into  the  knapsack  first  and  then 
the  next  most  beneficial.  This  continues  until  the  knapsack  is  full.  With  a  knapsack 
problem  though,  this  may  not  be  an  optimal  choice.  For  example,  items  one,  two,  and 
three  fill  the  knapsack.  However,  items  four  and  five  together  take  up  just  as  much  room 
as  item  three,  but  provide  more  benefit.  The  optimal  strategy  in  this  case  would  be  to 
have  items  one,  two,  four,  and  five  in  the  knapsack.  Therefore,  a  typical  knapsack 
problem  has  the  benefits  in  the  objective  function  and  the  constraint  contains  the  cost  or 
how  much  a  particular  item  fills  up  the  knapsack.  The  constraint’s  right  hand  side  is  the 
amount  of  space  available  in  the  knapsack.  Typically,  the  objective  function  is  maximized 
and  the  constraint  is  less  than  and  equal  to  some  value. 
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2.9  Summary 


This  chapter  provided  a  basic  understanding  of  how  NAIC  performs  its  resource 
allocation.  A  brief  review  was  then  conducted  to  discuss  how  other  military  and  civilian 
organizations  perform  their  resource  allocation.  An  overview  of  decision  analysis  and 
value-focused  thinking  as  a  methodology  used  for  NAIC’s  problem  was  then  explained 
with  some  background  into  decision  analysis  software.  Chapter  HI  applies  decision 
analysis  and  value-focused  thinking  to  NAIC’s  problem  and  shows  how  the  decision 
analysis  software  is  used  to  model  the  production  and  ranking  of  alternatives  for  NAIC’s 
resource  allocation  strategy. 
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in.  Decision  Analysis  Process  Using  Value-Focused  Thinking  for  NAIC 


3.1  Overview 

The  first  three  blocks  of  Figure  1  on  page  10  define  the  basic  structure  for  this 
chapter.  To  begin,  the  decision  situation  needs  to  be  identified  and  the  objectives  need  to 
be  understood.  The  decision  situation  involves  the  NAIC  Commander  choosing  a 
resource  allocation  strategy  from  a  set  or  list  of  alternatives  provided  by  the  SMT.  The 
overall  objective  is  to  minimize  the  impact  that  NAIC  will  incur  from  any  budget  cut  that 
this  organization  is  required  to  take.  To  best  meet  this  overall  objective,  value-focused 
thinking  was  used  to  develop  subobjectives  that  were  used  to  measure  the  impact  of 
alternatives  presented  to  the  commander.  These  subobjectives  are  developed  and  formed 
into  a  value  hierarchy.  The  commander  also  determines  the  relative  importance  of  those 
subobjectives  or  values  by  designating  weights  for  the  values. 

The  next  block  states  that  alternatives  need  to  be  identified.  This  section 
comprises  a  very  small  portion  of  this  chapter.  The  third  block  requires  considerable 
attention.  Most  of  Clemen’s  book  is  focused  on  decomposing  the  problem  [8;  7].  For  this 
block,  measures  are  introduced  and  defined.  DU  Managers  score  the  measures  using 
scoring  sheets.  Scoring  functions  are  then  used  to  translate  the  measures  from  the  scoring 
sheets  into  a  utility  score.  These  utility  scores  from  the  first  scoring  function  are  then 
aggregated  and  translated  into  another  utility  score.  After  this,  the  aggregated  scores  are 
multiplied  by  the  commander’s  weights  producing  scored  alternatives.  This  third  block 
also  incorporates  decision  analysis  software.  Using  DPL  and  Logical  Decisions,  the 
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problem  structure  and  preferences  are  modeled.  There  is  no  uncertainty  modeled.  The 
remaining  blocks  in  Figure  1  are  addressed  in  Chapters  IV  and  V. 


3.2  Understand  Objectives 

Developing  the  subobjectives  requires  an  understanding  of  the  overall  objective. 
The  overall  objective  is  to  minimize  the  impact  that  NAIC  will  incur  from  any  budget  cut 
that  it  is  required  to  take.  The  keyword  in  the  overall  objective  is  impact  and  that  forms 
the  basis  for  all  of  the  subobjectives.  In  other  words,  what  impacts  NAIC?  Before  this 
question  can  be  answered,  another,  more  important  question,  must  be  answered.  Who 
decides  what  impacts  NAIC?  Is  it  one  person  or  a  group  of  people  that  decide  this  issue? 
For  NAIC,  even  though  the  SMT  as  a  group  make  recommendations  to  the  commander 
for  NAIC's  resource  allocation  strategy,  the  commander  is  the  decision  maker  and  either 
rejects  or  accepts  their  recommendations. 

Section  3.2.1  discusses  the  subobjectives  chosen  by  the  NAIC  Commander  and 
places  those  subobjectives  in  a  value  hierarchy.  From  this  point  forward  in  the  discussion 
of  subobjectives,  the  subobjectives  are  given  the  term  values.  In  some  cases,  one  value  is 
just  as  important  as  another,  but  this  is  not  true  in  all  cases.  Section  3.2.2  gauges  the 
weight  of  one  value  over  another  in  order  to  determine  the  importance  of  the  values. 
Knowing  the  values  and  importance  of  those  values  completes  block  one  in  Figure  1, 
Understand  the  Objectives. 

The  NAIC  Commander’s  values  were  placed  in  a  value  hierarchy  and  then  each 
value  was  weighted.  Both  the  value  hierarchy  and  the  weights  are  very  critical  inputs  to 
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the  resource  allocation  model  and  are  shown  in  Appendix  A.  Defining  the  commander’s 
value  hierarchy  and  weighting  his  values  took  approximately  three  hours  of  the 
commander's  valuable  time  and  was  accomplished  in  two  separate  interviews  [1 1;  12]. 
Mrs.  Wilkinson  and  LTC  Kloeber  both  provided  important  information  during  these 
meetings  and  also  took  notes  to  ensure  the  accuracy  of  the  commander's  value  hierarchy 
and  associated  preferences. 


3.2.1  Values 

The  first  interview  with  the  NAIC  Commander  took  place  on  1  November  1996. 
The  purpose  of  the  interview  was  to  obtain  his  values  for  NAIC’s  resource  allocation 
strategy.  To  speed  up  the  process,  a  draft  value  hierarchy  was  developed  and  explained  to 
the  NAIC  Commander.  This  draft  drew  fundamental  values  from  NAIC’s  Mission  and 
Organizational  Pamphlet,  and  NAIC's  Master  Plan  [23;  24].  The  key  objectives  and  goals 
within  these  documents  provided  many  of  the  values  used  in  the  draft  hierarchy. 
Considerable  preliminary  coordination  occurred  to  determine  the  commander's  most 
probable  values.  During  the  interview,  it  was  very  important  to  ensure  the  commander 
understood  that  he  was  building  his  value  hierarchy  and  not  the  analyst’s.  He  could 
change  it  in  any  way  he  thought  necessary. 

In  particular,  the  commander  moved  the  Future  Requirements  value  to  a  much 
higher  level  and  took  away  the  Internal  Customer  value  in  order  to  focus  all  of  the  DUs  on 
the  External  Customer  value.  These  were  just  a  few  of  the  changes  he  made  and  the  entire 
process  took  approximately  one  hour.  The  commander  also  thought  that  further  research 
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was  needed  to  develop  the  Unit  Performance  portion  of  the  value  hierarchy.  Using  the 
Unit  Self  Assessment  and  the  Mission  Effectiveness  Inspection,  a  draft  value  hierarchy 
was  developed  [11;  29;  30].  This  value  hierarchy  was  then  sent  to  the  NAIC  Commander 
for  approval  and  was  returned  without  any  further  corrections. 

Figure  4  displays  the  top  three  levels  of  the  commander's  value  hierarchy.  The 


Figure  4.  Top  Three  Levels  of  Value  Hierarchy 


higher  the  value  is  in  the  tree,  the  more  important  the  value  is  to  the  commander.  As 
shown,  the  commander's  top  three  values  were  Customer  Support,  Future  Requirements, 
and  Unit  Performance.  The  three  values  under  the  Commander's  Values  block  also 
contain  values,  or  subobjectives,  under  them.  Remember  that  the  Commander’s  Values 
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block  is  the  overall  objective  which  is  to  minimize  the  impact  of  a  budget  cut  on  NAIC. 
Figure  4  shows  only  three  levels  in  the  value  hierarchy.  Overall,  there  are  five  levels  in  the 
value  hierarchy  which  is  presented  in  its  entirety  in  Appendix  A. 

3.2.2  Weights  or  Preferences 

The  second  interview  with  the  commander  was  conducted  on  20  December  1996 
[12].  This  interview  proved  to  be  much  more  time  consuming  and  demanding,  taking 
approximately  two  hours.  The  commander  weighted  all  of  the  values.  From  this 
interview,  some  values  were  deemed  unnecessary  for  this  resource  allocation  cycle  and 
some  were  combined  to  form  only  one  value.  For  example,  under  the  Future 
Requirements  value,  the  Futures  Analysis  value  was  comprised  of  Economics,  Leader 
Personality,  Military  Doctrine,  State  Relationships,  and  Technology  values.  These  five 
values  were  rolled  up  or  combined  into  the  Futures  Analysis  value  for  various  reasons.  To 
begin,  it  would  be  very  difficult  to  score  these  five  values  and  the  commander  felt  that  the 
DU  Managers  could  subjectively  consider  these  values  collectively  and  produce  an  overall 
score  for  all  five  values.  This  overall  score  was  placed  in  Futures  Analysis. 

The  commander  used  a  direct  assessment  method  to  weight  his  values.  The  direct 
assessment  was  based  on  the  following  thought: 

You  have  100  marbles.  These  marbles  can  be  distributed  in  any  manner  you  want. 
At  each  level  in  the  hierarchy  you  have  an  additional  100  marbles.  Once  aU  the 
marbles  are  gone,  there  are  no  more  for  that  level.  Consider  Customer  Support, 
Future  Requirements,  and  Unit  Performance.  How  would  you  distribute  those  100 
marbles? 
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There  is  a  distinction  between  branch  and  level.  The  top  value  is  the  overall  value,  or 
objective,  labeled  Commander’s  Value.  Under  this,  there  are  three  branches.  Customer 
Support,  Future  Requirements,  and  Unit  Performance  which  together  form  a  level.  The 
reason  they  form  a  level  is  that  all  three  have  the  same  value  above  them.  The  reason  they 
are  all  branches  is  that  each  have  levels  below  them.  Each  level  on  each  branch  was 
weighted  with  100  marbles.  The  commander  made  these  choices  based  on  what  he  thinks 
to  be  most  important.  For  instance,  look  at  Customer  Support  in  Figure  5.  Under 
Customer  Support  is  Acquisition,  Operational,  and  Policymakers  where  another  100 
marbles  were  distributed.  This  process  was  continued  for  each  branch  and  level. 
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Figure  5.  Top  Three  Levels  of  Value  Hierarchy  with  Weights 
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The  marbles  give  a  weighting  to  the  values  with  the  higher  the  number  having  more 
importance.  Looking  at  Figure  5,  one  can  see  that  the  Customer  Support/Operational 
holds  the  most  weight,  60*45,  then  Customer  Support/Acquisition  with  60*35,  then  Unit 
Performance/People  with  30*60,  and  so  on.  Note  that  Figure  5  only  contains  three  of  the 
five  levels.  An  observation  made  during  this  interview  was  that  the  commander  had  a 
more  difficult  time  scoring  four  or  more  values  on  one  level.  All  of  the  commander’s 
weights  are  given  in  the  value  hierarchy  in  Appendix  A. 


3.3  Identify  Alternatives 

The  decision  situation  is  for  the  commander  to  choose  a  resource  allocation 
strategy  and  the  objective  in  the  decision  situation  is  to  minimize  the  impact  on  NAIC  for 
NAIC’s  budget  cut.  Now,  according  to  Figure  1  on  page  10,  alternatives  must  be 
identified.  What  constitutes  an  alternative?  The  commander  chooses  a  resource  allocation 
strategy  based  on  a  certain  number  of  alternatives.  Since  the  objective  of  NAIC’s 
resource  allocation  strategies  is  to  minimize  the  impact  on  NAIC  of  any  budget  cut,  then 
each  alternative  for  this  methodology  is  based  on  how  it  impacts  NAIC.  The  impact  was 
developed  into  a  value  hierarchy  composing  the  commander's  values.  The  commander 
also  weighted  those  values.  So,  ultimately,  an  alternative  is  based  on  the  commander's 
values  and  weights  and  this  is  developed  in  greater  detail  later  in  this  chapter.  Now, 
remember  the  DU  Process.  There  are  19  DUs.  Each  DU  produces  plans  based  on  a  5%, 
10%,  15%,  and  20%  cut  level.  Thus,  each  DU  might  be  cut  0%,  5%,  10%,  15%,  or  20%. 
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Using  this  information,  the  resource  allocation  alternatives  for  NAIC  consist  of  each  DU 
taking  a  certain  percentage  cut  based  upon  how  those  cuts  impact  NAIC. 


3.4  Decompose  and  Model  the  Problem 

This  section  focuses  on  the  third  block  in  the  Decision- Analysis  Process  Rowchart 
shown  in  Figure  1.  According  to  Clemen,  "decomposition  is  the  key  to  decision  analysis" 
[8: 7].  NAIC's  problem  is  decomposed  and  broken  down  to  a  level  that  allows  the 
decision  maker  and  all  others  involved  with  this  problem  to  "understand  their  stmctures 
and  measure  uncertainty  and  value"  [8: 7].  While  there  is  no  uncertainty  involved  in  this 
problem,  understanding  NAIC's  values  is  very  important.  A  considerable  amount  of  work 
has  already  been  accomplished  with  values  in  Section  3.2  where  a  value  hierarchy  was 
formed  and  the  commander  determined  the  relative  importance  of  those  values  by 
assigning  weights  to  them.  The  remaining  portions  of  this  section  'decomposes'  the 
problem  at  hand.  While  an  alternative  consists  of  each  DU  taking  one  of  five  levels  of 
cuts,  this  section  goes  into  great  detail  on  how  those  cuts  are  scored.  Measures  are 
defined  from  the  value  hierarchy  and  a  scoring  sheet  is  produced  for  those  measures.  Two 
types  of  scoring  functions,  similar  to  utility  functions,  are  used  to  translate  unlike  units 
into  like  units.  Finally,  using  the  commander's  weights  that  were  assigned  to  his  values,  a 
score  for  an  alternative  is  developed. 
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3.4.1  Identify  Measures 


Measures  are  not  much  different  than  values.  To  be  precise,  measures  are 
measurable  values.  In  developing  the  value  hierarchy,  the  values,  or  subobjectives, 
continued  to  be  broken  down  into  more  values.  When  a  value  was  reached  that  could  no 
longer  be  broken  down  or  decomposed  into  more  values,  that  value  has  been  fathomed. 
For  example,  look  at  the  Unit  Performance  value  in  Figure  6. 


Figure  6.  Developing  Measures 


The  Processes  value  under  the  Unit  Performance  value  was  broken  down  into  Data, 
Planning,  Resources,  Safety,  and  Security  Concerns  values.  Data  and  Resources  were 
considered  capable  of  being  further  decomposed  whereas  Planning,  Safety,  and  Security 
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Concerns  values  could  not  be  broken  down  any  further.  Values  that  can  no  longer  be 
decomposed  become  candidates  for  measures.  It  is  not  automatic.  Some  candidates  for 
measures  are  very  difficult  to  gauge  on  some  scale.  Such  was  the  case  for  the  seven 
measures  under  Future  Requirements/IADS/Threat  Description  and  the  three  measures 
under  Future  Requirements/Info  Operations/Offense/(C4,  Electronic  Systems,  and  Space 
Systems).  These  measures  were  rolled  up  into  the  value  above  them.  The  value  that  they 
were  rolled  up  into  then  became  the  measure.  For  example,  Threat  Description,  C4, 
Electronic  Systems,  and  Space  Systems  became  the  measure.  The  measures  that  were 
rolled  up  are  shown  in  Appendix  A.  The  NAIC  Commander  also  changed  some  of  the 
other  measures.  For  instance,  at  one  point.  Relevance  and  Tailored  were  measures  under 
Customer  Support/(Acquisition,  Operational,  and  Policymakers)/Quality.  The  commander 
thought  there  was  little  distinction  between  these  two  measures  and  combined  them  into 
one  measure.  Relevance.  All  of  these  changes  occurred  during  the  second  interview  [12]. 

3.4.2.  Scoring  Measures 

In  Appendix  A,  the  measures  are  easily  recognized.  They  are  represented  by  ovals 
and  play  a  very  important  role  in  the  overall  model  because  they  are  the  actual  items  that 
the  DUs  score.  First  and  foremost,  the  DUs  used  a  scoring  sheet  to  rate  their  percentage 
cuts.  Once  these  scoring  sheets  were  marked,  these  sheets  were  used  to  find  the  utility 
score  for  each  plan  using  scoring  functions  for  each  measure.  There  are  41  measures  that 
need  to  be  scored  by  every  DU  Manager  for  each  level  of  cut. 
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3.4.2.1  Scoring  Sheets  for  Measures 


The  scoring  sheets  were  developed  using  line  scales  as  opposed  to  number  scales 
for  the  following  reasons.  Using  a  line  scale,  the  impact  that  a  certain  percentage  cut 
would  have  on  measures  is  stressed,  not  the  numerical  score  that  the  impact  would 
receive.  This  allows  the  DU  Managers  to  focus  on  the  impact  and  not  on  a  number.  If  a 
numbering  scale  was  used,  values  may  have  been  inflated.  Lastly,  line  scales  required  less 
time  to  score.  Whether  a  measure  should  receive  a  7.5  or  7.6  based  on  its  impact,  is  not 
considered  by  the  DU  Managers.  That  step  was  not  required  with  line  scales  and  saved 
time.  With  all  of  the  benefits  of  using  a  line  scale,  there  was  a  drawback.  While  the  time 
needed  to  score  the  percentage  cuts  was  less  for  the  DU  Managers,  the  analyst  is  required 
to  use  his/her  time  to  translate  the  impact  into  a  utility  score.  This  is  accomplished  using 
scoring  functions  for  the  measures  at  a  DU  level.  The  scoring  sheets  are  contained  in 
Appendix  B. 


3.4.2.2  Scoring  Functions  for  Individual  Measures 

For  NAIC,  there  are  two  applications  of  scoring  functions.  The  first  application 
was  applied  to  each  DU  on  all  41  measures.  The  scoring  functions  for  all  of  the  measures 
scored  using  the  first  application  were  linear  and  a  template  is  provided  in  Appendix  C. 
The  range  of  these  scoring  functions  goes  from  zero  to  ten  and  was  based  on  the  NAIC 
Commander's  recommendations  [12].  The  scoring  functions  could  have  had  any  range  as 
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long  as  the  range  was  consistent  [8:  474].  Again,  the  method  for  obtaining  a  particular 
measure's  utility  score  was  to  use  the  scoring  sheets  filled  out  by  the  DU  Managers.  Each 
line  scale  used  in  the  scoring  sheets  effectively  was  the  X-axis  for  the  scoring  functions. 
At  the  point  where  the  DU  Managers  placed  an  X  on  a  measure's  line  scale,  or  X-axis,  an 
orthogonal  line  was  drawn  up  to  the  point  of  intersection  on  the  scoring  function.  At  this 
point,  the  utility  score  on  the  Y-axis  was  obtained.  For  example,  consider  Figure  7.  This 
scoring  function  would  have  a  utihty  score  of  3.  This  type  of  scoring  was  accomplished 
for  each  measure,  for  each  percentage  cut,  for  each  DU.  Now,  the  aggregate  scoring 
functions,  or  the  second  appUcation  of  scoring  functions,  are  similar,  but  serve  a  different 
purpose  as  discussed  in  the  following  section. 


Figure  7.  Example  Scoring  Function  for  DU  Measures 
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3.4.2.3  Scoring  Functions  for  Aggregate  Measures 

With  the  first  application  of  the  scoring  functions  performed,  each  DU  has  scores 
for  their  measures  ranging  from  zero  to  ten  on  all  41  measures  for  each  cut  level.  The 
second  application  of  the  scoring  function  was  also  applied  to  all  41  measures,  but  in  a 
slightly  different  manner.  These  scoring  functions  were  used  to  obtain  an  overall  score  for 
an  alternative  where  the  first  application  of  scoring  functions  was  used  to  obtain  scores  for 
DU  plans.  Recall  that  an  alternative  consists  of  each  DU  taking  one  of  five  possible  cuts, 
0%,  5%,  10%,  15%,  or  20%.  Like  measures  were  then  added  together  across  the  DUs  for 
each  alternative.  To  help  understand  this,  consider  one  measure.  Timeliness.  The  first 
application  of  scoring  functions  has  been  accomplished.  Also  consider  one  alternative, 
where  the  cut  level  is  specified  for  each  of  the  19  DUs:  DUl  5%,  DU2  10%,  DU3  5%, 
DU4  20%,  DUS  0%,  DU6  10%,  DU7  0%,  DUS  20%,  DU9  15%,  DUIO  15%,  DUl  1  0%, 
DU12  20%,  DU13  0%,  DU14  5%,  DU15  15%,  DU16  10%,  DU17  0%,  DU18  20%  and 
DU 19  20%.  Since  the  first  application  of  scoring  functions  has  been  accomplished,  each 
of  the  DUs  at  the  given  level  of  cut  for  this  alternative  has  a  score  for  the  measure 
Timeliness.  These  19  distinct  scores  were  added  together  producing  a  second  score  that 
ranges  from  zero  to  190.  This  second  score,  which  is  marked  by  the  X  on  the  Timeliness 
axis  in  Figure  8,  was  then  used  for  the  second  application  of  scoring  functions  and  an 
impact  score  was  obtained.  This  process  was  performed  on  all  41  measures  for  this  single 
alternative.  The  Y-axis  has  a  range  from  0  to  10.  As  mentioned  earlier,  the  Y-axis  can 
have  any  range  as  long  as  there  is  consistency.  A  template  for  the  aggregate  scoring 
function  is  contained  in  Appendix  D. 
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Figure  8.  Example  Aggregate  Scoring  Function  for  Model 

3.5  Model  Summary 

Now  that  all  of  the  'pieces'  to  this  model  have  been  identified,  there  remains  the 
task  of  putting  the  puzzle  together.  One  of  the  pieces  was  put  together  already.  That 
piece  was  using  the  scoring  function  scores  for  (he  measures  from  the  DUs  and  adding 
them  together  to  assess  the  aggregate  scoring  function  to  obtain  an  impact  score.  The 
remaining  two  pieces,  the  commander's  value  hierarchy  and  the  commander's  weights, 
now  come  into  play.  Using  what  Clemen's  defines  as  an  additive  utility  function  and 
modified  for  the  extra  number  of  levels,  an  alternative  for  NAIC  was  scored  using 
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For  Equation  (1),  kw  represents  the  Commander’s  weight  for  the  wth  value,  kwx  represents 
the  Commander’s  weight  for  the  xth  value  under  the  ivth  value,  k^xy  represents  the 
Commander’s  weight  for  the  j’th  value  under  the  jcth  value  under  the  vvth  value,  and  so  on 
for  kwxyz-  W,  Xw,  Ywx,  and  Zwxy  are  the  number  of  branches  for  each  level  where  W  is  three 
based  on  the  Customer  Support,  Future  Requirements,  and  Unit  Performance  values. 
is  based  on  each  of  the  values  above  it.  Customer  Support,  Future  Requirements,  and  Unit 
Performance  where  the  branch  under  Customer  Support  has  Acquisition,  Operational  and 
Policymakers.  Therefore  Xi  is  three,  X2  is  five  because  of  Futures  Analysis,  IADS,  Info 
Operations,  New  Mission  Area,  and  Proliferation,  and  X3  is  two  because  of  Processes  and 
People.  Ywx  and  are  determined  in  a  similar  manner.  Uwxyz  is  the  scoring  function’s 
impact  score  for  the  measures  under  the  values.  The  upper  range  on  the  summation  signs 
ensure  the  summation  occurs  over  the  proper  value  [8:  557].  In  words,  this  states  that 
the  scoring  function’s  impact  score  for  the  aggregate  model  were  multiplied  by  the 
measure’s  weight  as  shown  in  the  value  hierarchy.  Those  values  were  then  multiplied  by 
the  weights  one  level  above  them  in  the  value  hierarchy.  Scores  are  then  added  together 
for  that  branch  and  level  and  then  multiplied  by  the  weight  on  the  next  level  up  from  them. 
This  continues  until  the  top  of  the  value  hierarchy  is  reached,  which  in  this  case  was 
Commander's  Values.  In  this  manner,  different  alternatives  earned  different  scores.  Once 
this  is  accomphshed,  the  best  alternative  with  respect  to  NAIC’s  value  hierarchy  can  be 
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chosen  to  complete  the  fourth  block  in  the  Decision- Analysis  Process  Flowchart  shown  in 
Figure  1  on  page  10.  There  is  one  item  that  must  be  discussed  before  moving  on  to  the 
software  used  to  model  this  problem.  The  additive  utility  function  must  meet  a  condition 
called  mutual  preferential  independence  in  order  to  be  applicable  to  NAIC's  value 
hierarchy. 

3.5.1  Mutual  Preferential  Independence 

This  concept  is  best  understood  using  an  example  taken  from  Clemen's  book  [8: 

579]. 

"An  attribute  Y  is  said  to  [be]  preferentially  independent  of  X  if  preferences 
for  specific  outcomes  of  Y  do  not  depend  on  the  level  of  attribute  X.  As  an 
example,  let  Y  be  the  time  to  completion  of  a  project  and  X  its  cost.  If  we  prefer  a 
project  time  of  5  days  to  one  of  10  days,  assuming  that  the  cost  is  100  in  each 
case,  and  if  we  also  prefer  a  project  time  of  5  days  to  one  of  10  days  if  the  cost  is 
200  in  both  cases,  then  Y  is  preferentially  independent  of  X;  it  does  not  matter 
what  the  cost  is~we  still  prefer  the  shorter  completion  time." 

In  order  to  have  mutual  preferential  independence  in  the  above  example,  cost  must  also  be 
preferentially  independent  of  project  time.  Clemen  gives  an  example  of  where  mutual 
preferential  independence  may  not  hold  by  describing  how  ones  "preference  for  amount  of 
homework  effort  might  depend  on  course  topic"  or  even  the  instructor  teaching  the 
material  [8;  579].  The  question  now  turns  to  whether  or  not  NAIC's  additive  utility 
function  has  mutual  preferential  independence  as  applied  to  their  problem.  Clemen 
discusses  how  one  can  determine  whether  or  not  independence  exists  for  NAIC's  problem 
and  says  the  simplest  approach  is  to  use  paired  comparisons  [8:  580].  Since  this  is  only  a 
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first  look  at  NAIC's  problem,  mutual  preferential  independence  was  not  checked;  it  was 
assumed  to  hold.  Clemen  says  that  "if  a  decision  maker  has  done  a  good  job  of  building  a 
decomposable  hierarchy,  mutual  preferential  independence  probably  is  a  reasonable 
assumption"  [8:  579].  A  decomposable  hierarchy  has  been  built  for  NAIC.  Assuming 
mutual  preferential  independence  is  appropriate  at  this  point  for  NAIC's  problem. 


3.6  Software  Used  to  Model  the  Problem 

Using  software  to  model  the  problem  falls  under  block  three  in  the  Decision- 
Analysis  Process  Flowchart  in  Figure  1.  While  some  background  in  DPL  and  LDW  has 
already  been  given  in  Chapter  H,  this  section  shows  how  the  software  was  used  and 
applied  to  this  problem.  Various  pieces  of  software  were  used  to  model  the  problem.  The 
value  hierarchy  was  the  basic  building  block  for  the  model.  Without  the  values,  the  model 
has  no  other  basis.  The  weights  for  the  measures  were  incorporated  into  the  decision 
analysis  model.  After  the  line  scales  from  the  scoring  sheets  were  converted  to  a  utility 
score,  these  scores  for  the  measures  were  placed  into  an  Excel  Spreadsheet  and  read  into 
DPL.  Excel  was  used  because  it  provided  quicker  access  to  the  scores.  DPL 
automatically  adds  the  measures  designated  as  attributes  for  an  alternative  from  one  DU  to 
the  next  producing  the  aggregated  values.  Once  all  of  the  DUs’  measures,  with  each  DU 
at  a  certain  percentage  cut  level,  were  added,  the  second  scoring  function  was  applied. 
Using  Equation  (1),  the  weights  were  applied  to  the  scoring  functions  and  an  alternative 
was  scored.  Budget  constraints  were  used  to  decrease  the  run  time.  The  following 
sections  go  into  much  greater  detail  about  the  model  that  was  built.  Chapter  IV  contains 
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the  results  where  the  best  alternative  was  chosen  and  sensitivity  analysis  was  performed. 
Chapter  V  addresses  the  final  two  blocks  of  Figure  1  by  discussing  whether  or  not  further 
analysis  is  needed.  In  addition,  the  implications  of  implementing  the  chosen  alternative 
and  recommendations  for  future  work  are  provided  in  Chapter  V. 


3.6.1  DPL®  3.20.01 

DPL  was  the  core  model  building  software  [10].  There  were  two  areas  in  DPL 
that  were  important  to  the  NAIC  model,  the  influence  diagram  and  the  decision  tree.  The 
first  area  is  the  influence  diagram.  The  model  used  two  groups  of  figures  in  the  influence 
diagram.  The  first  group  contained  the  decision  to  be  made  by  each  DU  and  the  second 
contained  the  commander's  weights  which  were  displayed  in  a  value  hierarchy  format. 

The  first  diagram  is  shown  in  Appendix  E.  The  decision  tree  was  automatically  generated 
from  the  influence  diagram.  However,  the  decision  tree  was  modified  significantly  in 
order  to  best  model  NAIC’s  problem.  Both  the  influence  diagram  and  decision  tree 
provided  various  key  requirements  that  allowed  the  model  to  be  effective. 

3.6.1.1  Influence  Diagram 

There  were  19  decision  nodes  in  the  influence  diagram.  Each  decision  node 
contained  five  alternatives  or  choices.  The  numerical  value  placed  on  those  branches 
corresponded  to  the  percentage  cut  taken  for  that  DU.  The  numerical  values  on  those 
branches  are  pointers  to  an  array  in  Excel  which  is  described  later.  The  data  from  the 


34 


spreadsheet  was  converted  into  DPL  code.  Now,  DPL  was  told  to  use  that  particular  data 
by  selecting  the  Include  option  and  then  selecting  Before  Data  Definitions  option.  If  this 
was  not  accomplished,  the  program  would  not  run  correctly.  With  these  steps 
accomplished,  several  steps  were  then  taken  in  the  decision  tree  view. 


3.6.1.2  Decision  Tree 

The  decision  tree  view  contained  several  items  that  needed  to  be  accomplished.  In 
this  view,  44  attributes  were  defined.  These  attributes  can  be  viewed  as  the  variables  for 
the  objective  function  and  constraints.  Attributes  were  used  to  track  all  41  measures 
which  were  used  to  develop  the  objective  function.  The  remaining  three  attributes  were 
used  as  variables  for  two  budget  constraints.  The  constraints  are  necessary  because  the 
run  time  is  dramatically  decreased  when  the  constraints  are  used  to  prune  some  of  the 
branches  of  the  tree.  For  example,  a  13  DU  model’s  run  time  was  decreased  from 
approximately  200  hours  to  five  hours  using  the  constraints.  The  attributes  are  shown  in 
Appendk  F  and  the  constraints  and  objective  function  are  shown  in  Appendix  G. 

3.6.1.2.1  Attributes 

There  were  41  attributes  used  for  the  objective  function,  representing  the  41 
measures,  and  three  attributes  used  for  two  constraints  for  a  total  of  44  attributes.  The 
attributes  were  designated  in  the  get/pay  expressions  for  a  decision  node  in  DPL.  An 
undocumented  function  of  DPL  allows  attributes  to  automatically  add  together  as 
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progress  was  made  from  one  decision  node  to  the  next.  Also,  this  model  extensively  made 
use  of  arrays  within  DPL.  There  was  currently  no  documented  instruction  for  how  this 
would  be  accomplished,  but  ADA  technical  support  staff  assisted  with  this  effort  [7].  The 
last  item  that  needed  clarification  was  the  second  to  last  attribute  which  contained  a  minus 
sign.  This  attribute  was  used  to  help  keep  track  of  the  amount  of  money  cut  from  an 
individual  DU  at  a  particular  level  and  played  a  key  role  in  one  of  the  constraints.  The 
specific  reason  the  minus  sign  was  needed  was  to  discount  the  automatic  addition  that  was 
referred  to  earlier.  There  was  not  a  method  to  'turn  off  the  automatic  addition  so  this 
aspect  was  hard  coded  in  order  to  turn  it  off.  Since  the  first  decision  node  had  nothing 
added  to  it,  it  was  not  necessary  to  subtract  anything.  The  attributes  for  each  section  of 
the  decision  tree  are  contained  in  Appendix  F. 


3.6. 1.2.2  Constraints 

Two  constraints  served  a  very  important  purpose.  As  already  stated,  the  program 
time  was  cut  dramatically  using  the  two  constraints.  A  concept  that  must  be  included  here 
was  that  constraints  in  DPL  were  evaluated  at  each  decision  node.  The  constraints  were 
built  on  the  actual  amount  of  budgetary  cuts  that  NAIC  was  required  to  perform.  Given  a 
certain  dollar  amount  that  NAIC  had  to  cut,  the  first  constraint  was  based  on  going  over 
this  cut  amount  and  the  second  on  going  under  this  cut  amount.  The  thought  process  for 
these  constraints  was  simple.  To  begin,  the  19  DUs  were  rank  ordered  by  the  amount  of 
money  they  had  available  for  cutting,  highest  to  lowest.  Then,  using  DPL's  ability  to  add 
attributes  as  it  goes  along  in  the  decision  tree  and  that  constraints  were  checked  at  each 
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decision  node,  the  first  variable  of  the  constraint  contained  the  actual  dollar  amount  cut  so 


far  for  that  alternative  up  to  that  point.  Added  to  this  variable  was  the  total  summed 
amount  of  dollar  cuts  for  the  remaining  DUs  set  at  the  highest  level  of  cuts.  This  provided 
a  value  that  could  be  compared  to  the  required  dollar  amount  that  NAIC  needed  to  cut.  If 
this  constraint  was  violated,  then  the  branch  was  pruned  from  that  point  forward. 

The  lower  constraint  was  similar.  Instead  of  not  cutting  enough  however,  it  is  also 
possible  to  cut  too  much.  Setting  this  constraint  reasonably  over  the  dollar  amount  that 
NAIC  was  required  to  cut  provided  a  reasonable  bound.  If  an  alternative,  at  any  point  in 
the  decision  tree,  went  over  this  amount,  that  branch  was  also  halted  and  no  longer 
evaluated.  So,  upper  and  lower  bounds  were  put  on  the  dollar  amount  cut.  If  a  branch 
evaluated  at  a  certain  point  in  the  decision  tree  did  not  fall  within  these  bounds,  it  was 
pruned.  The  constraint  functions  are  given  in  Appendix  G. 


3.6. 1.2.3  Objective  Function 

The  objective  function  also  contained  the  same  two  constraints.  The  reasoning 
was  that  the  objective  function  was  evaluated  at  the  very  end  of  the  decision  tree.  The 
constraint  functions  were  evaluated  only  at  the  current  node  and  did  not  include  evaluation 
at  the  end  node.  So,  in  order  for  the  constraints  to  still  apply,  they  were  included  in  the 
objective  function.  The  objective  function  was  formed  using  Equation  (1)  from  Section 
3.3.  Incorporating  the  weights  that  were  set  in  the  influence  diagram  and  the  scoring 
functions  for  the  individual  measures  into  Equation  (1),  the  objective  function  was 
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produced.  The  objective  was  to  maximize  Commander's  Values  and  the  objective 
equation  used  to  do  it  is  shown  in  Appendix  G. 


3.6.2  Excel®  7.0 

Excel  was  used  as  a  storage  and  input  device  [22].  It  stored  the  DU  level  utility 
scores  for  each  DU  for  each  measure  for  each  percentage  cut  in  a  5x44  array.  The  five 
rows  were  associated  with  the  different  percentage  cuts  and  the  44  columns  were 
associated  with  the  41  measures  and  3  budget  constraint  variables.  These  arrays  were 
then  converted  into  DPL  code  fi'om  the  Main  DPL  Program  window  and  used  in  the  DPL 
model.  The  spreadsheet  values  are  contained  in  Appendix  H. 


3.6.3  Logical  Decisions®  4.01 

Logical  Decisions  was  used  preliminarily  to  build  the  value  hierarchy  [17].  The 
hierarchy  built  in  Logical  Decisions  is  contained  in  Appendix  I.  Logical  Decision  was 
considered  a  good  software  package  to  perform  some  sensitivity  analysis  on  the  top 
twenty  alternatives  and  as  a  means  to  score  and  generate  the  DU  level  utility  functions. 


3.7  Summary 

This  chapter  went  into  great  detail  on  the  first  three  blocks  of  the  Decision- 
Analysis  Process  Flowchart.  An  NAIC  value  hierarchy  was  developed,  the  commander 
weighted  those  values,  measures  were  defined,  scoring  sheets  were  made,  and  scoring 
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functions  were  defined.  Also,  software  was  used  to  implement  and  make  use  of  all  of 
these  pieces  of  information.  Chapter  IV  discusses  some  changes  that  occurred  over  the 
course  of  this  study  which  prompted  some  elements  from  Chapter  III  to  be  modified. 
While  the  value  hierarchy,  weights,  measures,  scoring  sheets,  and  scoring  functions 
remained  mostly  unchanged,  the  model  built  using  the  software  was  no  longer  applicable. 
This  points  to  the  usefulness  of  value-focused  thinking.  While  the  whole  situation  or 
model  may  change,  the  ingredients  of  the  model  are  very  robust  and  will  not  change  unless 
a  new  decision  maker  comes  along  or  the  previous  decision  maker  deems  it  necessary. 
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rv.  Analysis  and  Results  for  NAIC  (FY99  Cycle) 


4.1  Overview 

The  FY99  Budget  Cycle  at  NAIC  forced  some  changes  in  the  methodology 
presented  in  Chapter  ni.  While  NAIC  was  forced  to  take  a  13%  cut,  the  commander 
decided  to  take  that  cut  entirely  from  external  assistance,  research  and  development,  and 
procurement.  For  validation  purposes,  only  Tier  HI  program  items  (external  assistance 
proposals)  are  evaluated  in  this  thesis.  As  such,  there  were  62  proposals  to  evaluate  by 
ten  DU  Managers  where  the  proposals  would  be  cut  completely  or  entirely  saved.  NAIC 
wanted  these  62  proposals  ranked  using  the  methodology  presented  in  Chapter  IB. 


4.2  NAIC  Requirements 

The  methodology  was  general  enough  to  account  for  the  change  in  budgeting 
procedures.  The  value  hierarchy  still  applies  as  do  the  commander's  weights.  This  is  one 
of  the  strengths  of  value-focused  thinking.  It  can  be  applied  in  many  different  cases  and 
used  for  various  studies  or  analyses.  The  scoring  sheets  required  a  few  changes.  Wording 
of  the  measures  changed  from  ‘a  certain  percentage  cut’  to  ‘if  this  contract  is  cut.’  There 
was  only  one  line  scale  for  each  measure  instead  of  a  line  scale  for  5%,  10%,  15%,  and 
20%  because  the  proposals  would  either  be  cut  or  not  cut.  Also,  13  of  the  measures  were 
not  included  in  the  scoring  sheets.  NAIC  analysts  thought  these  13  measures  did  not 
apply  and  would  not  produce  any  change  in  the  scores  to  warrant  their  use.  The  13 
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measures  left  out  were  the  three  DoDIPP  measures.  Promotion  Frontier,  Communication, 


Work  Environment,  Planning,  Safety,  Security  Concerns,  Analyze,  Manage,  Equipment, 
and  Facility.  While  this  saved  some  time  for  the  ten  DU  Managers  that  scored  the 
measures,  time  savings  may  have  hindered  obtaining  a  complete  overall  utility  score. 
Added  thought  should  have  been  given  to  scoring  or  leaving  out  these  13  measures. 
Equation  (1)  also  still  applied  to  the  evaluation  of  the  62  proposals  and  was  used  to  rank 
the  proposals. 

Instead  of  providing  NAIC  a  certain  set  of  alternatives  for  their  resource  allocation 
strategy,  NAIC  wanted  the  62  proposals  scored  and  ranked  using  the  value  hierarchy, 
commander's  weights,  scoring  sheets,  and  the  first  set  of  scoring  functions.  At  this  point, 
NAIC,  based  on  the  cut  required,  would  designate  the  proposals  to  be  cut  based  on  the 
rank  order.  This  presents  some  problems  because  it  is  highly  unlikely  that  the  required 
amount  of  resources  will  permit  cutting  an  integer  number  of  proposals.  So,  to  help  in  this 
endeavor,  some  cost-benefit  analysis  is  performed  and  a  model  can  be  built  in  the  same 
fashion  as  in  Chapter  in.  Chapter  Ill's  model  contained  19  DUs  with  five  levels  each. 

The  model  for  this  current  stmcture  would  contain  62  proposals  with  two  levels  each  and 
produce  resource  allocation  alternatives  for  NAIC,  not  ranking  of  the  proposals. 


4.3  Analysis 

This  section  focuses  on  the  results  obtained  from  the  methodology  used  for 
NAIC’s  resource  allocation  strategy.  First,  the  results  for  the  62  proposals  are  examined 
and  ranked  solely  on  the  score  they  received  using  the  value  hierarchy,  commander’s 
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weights,  scoring  sheets,  and  scoring  functions  combined  to  produce  an  overall  score. 
Next,  the  individual  cost  of  each  proposal  is  incorporated  and  a  cost-benefit  analysis  is 
provided,  based  on  the  computed  overall  score.  In  section  4.3,  sensitivity  analysis  is 
performed  on  the  weights  assigned  to  the  top  three  values  under  the  overall  value. 
Commander’s  Values.  The  three  weights  examined  are  for  the  values.  Customer  Support, 
Future  Requirements,  and  Unit  Performance.  Finally,  the  actual  policy  that  NAIC 
implemented  for  this  cycle  is  examined  and  compared  to  the  results  attained  in  this 
chapter. 


4.3.1  Results  Based  on  Methodology 

The  results  in  Table  1  rank  the  62  proposals  for  external  assistance  based  on  the 
methodology  set  forth  in  Chapter  in.  The  methodology  combined  the  NAIC 
Commander's  value  hierarchy,  the  NAIC  Commander's  weights,  the  measures  and  their 
scoring  sheets,  and  the  utility  scores  from  the  scoring  sheets.  These  ingredients  were  used 
to  determine  an  overall  impact  score  for  each  proposal  based  on  the  overall  objective  to 
minimize  the  impact  of  a  budget  cut  to  NAIC.  A  low  score  indicates  a  low  impact.  With 
this  in  mind,  the  overall  strategy  is  to  select  for  cutting  the  proposals  having  the  lowest 
impact  until  the  necessary  amount  of  budgetary  cut  is  achieved.  Table  1  ranks  the 
proposals  based  on  their  impact.  It  also  shows  the  funding  level  for  each  proposal  as  well 
as  the  cumulative  impact  and  cumulative  amount  cut  for  any  point  in  the  table.  NAIC 
took  a  $8,544  million  cut  for  this  cycle.  Using  Table  1  and  drawing  the  line,  which  is 
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shown  at  $8,789  million,  results  in  cutting  22  proposals  with  a  cumulative  impact  of 
36.6628. 


Table  1.  Rank  Order  based  on  Impact 


RANK 

ORDER 

PROPOSAL 

NUMBER 

IMPACT 

SCORE 

CUMULATIVE 

IMPACT 

FUNDING 

(1000) 

CUMULATIVE 
AMOUNT  CUT 

1 

DU9-13 

0.435 

0.435 

233 

233 

2 

DU9-1 

0.478 

0.913 

173 

406 

3 

DU9-9 

0.837 

1.750 

326 

732 

4 

DU9-4 

0.884 

2.633 

281 

1013 

5 

DU9-2 

1.066 

3.699 

384 

1397 

6 

DU6-4 

1.123 

4.822 

218 

1615 

7 

DU3-6 

1.199 

6.021 

298 

1913 

8 

DU9-18 

1.232 

7.253 

187 

2100 

9 

DU9-19 

1.402 

8.655 

93 

2193 

10 

DU6-1 

1.509 

10.164 

130 

2323 

11 

DU9-15 

1.591 

11.756 

359 

2682 

12 

DU9-10 

1.839 

13.594 

698 

3380 

13 

DU9-14 

1.971 

15.565 

716 

4096 

14 

DU9-16 

2.041 

17.606 

932 

5028 

15 

DU9-21 

2.050 

19.656 

154 

5182 

16 

DU9-20 

2.125 

21.781 

218 

5400 

17 

DU9-7 

2.255 

24.036 

417 

5817 

18 

DU9-17 

2.279 

26.315 

1396 

7213 

19 

DU6-5 

2.401 

28.716 

70 

7283 

20 

DUl-2 

2.610 

31.326 

445 

7728 

21 

DUl-5 

2.614 

33.940 

130 

7858 

22 

DU9-8 

2.723 

36.663 

931 

8789 

23 

DU9-6 

2.736 

39.399 

182 

8971 

24 

DU6-3 

2.741 

42.140 

247 

9218 

25 

DU9-5 

2.959 

45.099 

696 

9914 

26 

DU9-3 

2.986 

48.086 

714 

10628 

27 

DU3-4 

3.200 

51.286 

135 

10763 

28 

DUl-6 

3.214 

54.500 

211 

10974 

29 

DU3-3 

3.228 

57.728 

671 

11645 

30 

DUl-3 

3.247 

60.975 

495 

12140 

31 

DU5-1 

3.285 

64.260 

524 

12664 

32 

DUl-1 

3.288 

67.548 

390 

13054 

43 


33 

DU9-11 

3.317 

70.865 

465 

13519 

34 

DU6-7 

3.348 

74.213 

257 

13776 

35 

DU6-6 

3.382 

77.595 

24 

13800 

36 

DU5-2 

3.529 

81.124 

233 

14033 

37 

DU5-3 

3.633 

84.757 

322 

14355 

38 

DU4-5 

3.812 

88.569 

110 

14465 

39 

DU6-2 

3.834 

92.403 

512 

14977 

40 

DU3-1 

3.908 

96.311 

699 

15676 

41 

DU9-12 

3.970 

100.281 

698 

16374 

42 

DU8-1 

4.078 

104.359 

933 

17307 

43 

DU8-3 

4.326 

108.684 

87 

17394 

44 

DUlO-2 

4.438 

113.122 

775 

18169 

45 

DU8-5 

4.469 

117.592 

1834 

20003 

46 

DU8-4 

4.558 

122.150 

372 

20375 

47 

DU4-2 

4.561 

126.711 

342 

20717 

48 

DU4-3 

4.576 

131.287 

378 

21095 

49 

DU3-2 

4.628 

135.916 

567 

21662 

50 

DU3-5 

4.757 

140.673 

758 

22420 

51 

DU7-1 

4.938 

145.611 

347 

22767 

52 

DU4-6 

4.968 

150.579 

735 

23502 

53 

DU4-4 

5.003 

155.582 

192 

23694 

54 

DUlO-1 

5.024 

160.606 

463 

24157 

55 

DU2-3 

5.232 

165.838 

311 

24468 

56 

DU2-2 

5.234 

171.073 

218 

24686 

57 

DU8-2 

5.250 

176.323 

87 

24773 

58 

DU4-1 

5.372 

181.695 

110 

24883 

59 

DU2-1 

5.377 

187.071 

208 

25091 

60 

DU2-4 

5.395 

192.466 

903 

25994 

61 

DU4-7 

5.507 

197.973 

261 

26255 

62 

DU3-7 

5.534 

203.507 

791 

27046 

4.3.2  Results  Based  on  Cost-Benefit  Analysis 

Another  way  to  rank  the  62  proposals  is  by  impact-cost  ratio  which  is  impact 
divided  by  the  cost.  As  a  side  note,  cost  could  have  been  divided  by  the  benefit,  but  since 
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the  objective  has  been  to  minimize,  that  concept  is  continued.  Table  2  shows  these 
rankings.  For  the  $8,544  million  that  NAIC  cut,  drawing  the  line  at  $8,916  million  results 
in  cutting  13  proposals  with  a  cumulative  impact  of  23.206. 


Table  2.  Rank  Order  based  on  Impact/Cost 


RANK 

ORDER 

PROPOSAL 

NUMBER 

IMPACT 

IMPACT 

SCORE 

CUMULATIVE 

IMPACT 

FUNDING 

(1000) 

CUMULATIVE 
AMOUNT  CUT 

COST 

1 

DU9-17 

0.00163 

2.279 

2.279 

1396 

1396 

2 

DU9-13 

0.00187 

0.435 

2.714 

233 

1629 

3 

DU9-16 

0.00219 

2.041 

4.755 

932 

2561 

4 

DU8-5 

0.00244 

4.469 

9.224 

1834 

4395 

5 

DU9-9 

0.00257 

0.837 

10.061 

326 

4721 

6 

DU9-10 

0.00263 

1.839 

11.900 

698 

5419 

7 

DU9-14 

0.00275 

1.971 

13.871 

716 

6135 

8 

DU9-1 

0.00276 

0.478 

14.349 

173 

6308 

9 

DU9-2 

0.00278 

1.066 

15.414 

384 

6692 

10 

DU9-8 

0.00292 

2.723 

18.137 

931 

7623 

11 

DU9-4 

0.00314 

0.884 

19.021 

281 

7904 

12 

DU3-6 

0.00402 

1.199 

20.220 

298 

8202 

13 

DU9-3 

0.00418 

2.986 

23.206 

714 

8916 

14 

DU9-5 

0.00425 

2.959 

26.165 

696 

9612 

15 

DU8-1 

0.00437 

4.078 

30.243 

933 

10545 

16 

DU9-15 

0.00443 

1.591 

31.834 

359 

10904 

17 

DU3-3 

0.00481 

3.228 

35.063 

671 

11575 

18 

DU6-4 

0.00515 

1.123 

36.186 

218 

11793 

19 

DU9-7 

0.00541 

2.255 

38.441 

417 

12210 

20 

DU3-1 

0.00559 

3.908 

42.349 

699 

12909 

21 

DU9-12 

0.00569 

3.970 

46.319 

698 

13607 

22 

DUlO-2 

0.00573 

4.438 

50.757 

775 

14382 

23 

DUl-2 

0.00586 

2.610 

53.366 

445 

14827 

24 

DU2-4 

0.00597 

5.395 

58.761 

903 

15730 

25 

DU5-1 

0.00627 

3.285 

62.047 

524 

16254 

26 

DU3-5 

0.00628 

4.757 

66.804 

758 

17012 

27 

DUl-3 

0.00656 

3.247 

70.051 

495 

17507 

28 

DU9-18 

0.00659 

1.232 

71.282 

187 

17694 

45 


29 

DU4-6 

0.00676 

4.968 

76.251 

735 

18429 

30 

DU3-7 

0.00700 

5.534 

81.785 

791 

19220 

31 

DU9-11 

0.00713 

3.317 

85.101 

465 

19685 

32 

DU6-2 

0.00749 

3.834 

88.935 

512 

20197 

33 

DU3-2 

0.00816 

4.628 

93.564 

567 

20764 

34 

DUl-1 

0.00843 

3.288 

96.852 

390 

21154 

35 

DU9-20 

0.00975 

2.125 

98.976 

218 

21372 

36 

DUlO-1 

0.01085 

5.024 

104.000 

463 

21835 

yi 

DU6-3 

0.01110 

2.741 

106.741 

247 

22082 

38 

DU5-3 

0.01128 

3.633 

110.375 

322 

22404 

39 

DU6-1 

0.01161 

1.509 

111.884 

130 

22534 

40 

DU4-3 

0.01211 

4.576 

116.460 

378 

22912 

41 

DU8-4 

0.01225 

4.558 

121.018 

372 

23284 

42 

DU6-7 

0.01303 

3.348 

124.366 

257 

23541 

43 

DU9-21 

0.01331 

2.050 

126.416 

154 

23695 

44 

DU4-2 

0.01334 

4.561 

130.977 

342 

24037 

45 

DU7-1 

0.01423 

4.938 

135.915 

347 

24384 

46 

DU9-6 

0.01504 

2.736 

138.652 

182 

24566 

47 

DU9-19 

0.01507 

1.402 

140.054 

93 

24659 

48 

DU5-2 

0.01515 

3.529 

143.582 

233 

24892 

49 

DU  1-6 

0.01523 

3.214 

146.796 

211 

25103 

50 

DU2-3 

0.01682 

5.232 

152.029 

311 

25414 

51 

DUl-5 

0.02011 

2.614 

154.643 

130 

25544 

52 

DU4-7 

0.02110 

5.507 

160.150 

261 

25805 

53 

DU3-4 

0.02371 

3.200 

163.350 

135 

25940 

54 

DU2-2 

0.02401 

5.234 

168.585 

218 

26158 

55 

DU2-1 

0.02585 

5.377 

173.961 

208 

26366 

56 

DU4-4 

0.02606 

5.003 

178.964 

192 

26558 

57 

DU6-5 

0.03431 

2.401 

181.365 

70 

26628 

58 

DU4-5 

0.03465 

3.812 

185.177 

no 

26738 

59 

DU4-1 

0.04884 

5.372 

190.549 

no 

26848 

60 

DU8-3 

0.04972 

4.326 

194.875 

87 

26935 

61 

DU8-2 

0.06035 

5.250 

200.125 

87 

27022 

62 

DU6-6 

0.14092 

3.382 

203.507 

24 

27046 
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4.3.3  Optimal  Strategy 


While  the  alternative  selected  based  on  the  Impact/Cost  had  a  much  lower 
cumulative  impact  than  the  alternative  selected  based  on  the  ranked  impacts  alone,  this 
may  still  not  be  the  optimal  alternative.  To  set  the  stage  for  this  optimization,  NAIC  must 
cut  at  least  $8,544  million.  Also,  each  proposal  is  either  cut  completely  or  not  at  all. 

There  is  a  benefit,  or  impact,  associated  with  each  proposal.  These  items  fill  the  necessary 
requirements  to  be  treated  as  a  knapsack  problem.  The  objective  function  in  this  case 
would  be  to  minimize  impact  where  each  proposals  impact  is  multiplied  by  one  or  zero 
based  on  whether  or  not  that  proposal  is  chosen  to  be  cut.  The  sole  constraint  contains 
the  cost  for  each  proposal  and  it  is  also  multiplied  by  zero  or  one  for  the  same  reason. 

The  constraint  is  greater  than  or  equal  to  $8,544  million  because  this  amount  must  be  cut. 
The  formulation  becomes: 

minimize  biXi 

subject  to:  CiX,  >  8544 

Xi  =  0or  1,  i  =  1,...,62 

where  bi,  Ci,  and  x,  is  the  ith  proposal’s  impact  score,  funding  level  (1000),  and 
whether  the  proposal  is  cut  (xi=l)  or  not  (xi=0)  respectively. 

Coding  this  problem  into  LINDO  yielded  an  optimal  solution  where  the  proposals 
DU9-17,  DU9-13,  DU9-16,  DU8-5,  DU9-9,  DU9-10,  DU9-14,  DU9-1,  DU9-2,  DU9-8, 
DU9-4,  and  DU93  were  selected  to  be  cut  [19].  The  only  difference  between  this 
alternative  and  the  alternative  selected  using  the  Impact/Cost  is  that  the  proposal  DU3-6 
was  not  selected.  Also,  the  LINDO  optimal  solution  cuts  $8,618  million  instead  of  $8,916 
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million  and  only  12  proposals  instead  of  13.  The  cumulative  impact  drops  to  22.007.  The 
LINDO  code  and  variables  used  to  obtain  those  codes  are  contained  in  Appendix  J. 


4.3.4  Selecting  Alternatives 

Selecting  alternatives  satisfies  the  fourth  block  in  the  Decision-Analysis  Process 
Flowchart  by  allowing  NAIC  to  choose  the  best  alternative  from  Table  1,  Table  2  or  the 
Knapsack  problem.  The  means  for  choosing  an  alternative  is  fairly  straightforward  using 
Table  1  or  Table  2  and  the  alternative  is  already  provided  using  LINDO.  Given  a  required 
dollar  amount  to  cut,  NAIC  looks  at  the  far  right  column  in  either  Table  1  or  Table  2  and 
finds  the  dollar  amount  that  exceeds  the  amount  to  cut.  At  this  point,  all  proposals  above 
this  line  are  the  proposals  to  cut.  A  few  items  need  to  be  considered  at  this  point.  First, 
the  objective  is  to  minimize  total  impact  to  NAIC.  With  this  in  mind.  Table  2  would  be 
chosen  over  Table  1  because  for  any  dollar  amount  cut  up  to  the  very  last  proposal.  Table 
2  has  less  impact  for  NAIC.  The  LINDO  solution  would  be  chosen  over  Table  2  because 
less  impact  is  cut.  However,  the  LINDO  solution  may  be  more  difficult  to  explain  and  the 
impact  savings  may  not  outweigh  this. 

The  second  item  that  needs  to  be  considered  in  selecting  an  alternative,  is  how  far 
over  the  cut  line  NAIC  is  willing  to  go.  For  example,  consider  Table  1.  Also  consider 
that  NAIC  is  required  to  take  a  $5.8 18  million  cut.  This  is  a  hard  constraint  and  $5,818 
million  must  be  cut.  However,  in  Table  1,  up  to  and  including  Proposal  17  cuts  $5,817 
million  and  Proposal  18  cuts  $1,396  million  by  itself.  Using  the  method  set  forth  in  the 
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previous  paragraph,  $1,395  million  over  the  required  amount  is  cut.  However,  if  Proposal 
19  is  cut  instead  of  Proposal  18,  only  $.069  million  over  the  required  amount  is  cut. 

Lastly,  consider  the  proposals  belonging  to  DU9.  In  Table  1, 19  of  their  21 
proposals  were  in  the  top  30  with  low  impact  scores.  In  Table  2,  16  of  their  21  proposals 
were  in  the  top  30  with  low  impact-cost  scores.  This  may  represent  a  problem.  If  the 
commander  does  not  want  to  cut  a  high  proportion  of  the  number  of  proposals  from  DU9, 
then  the  tables  can  be  altered.  When  the  number  of  proposals,  based  on  cost,  impact  or 
some  other  criteria,  is  reached,  the  remaining  proposals  for  DU9  are  taken  out  of  the  table 
and  all  other  proposals  are  moved  up.  This  can  be  done  for  any  DU. 

To  conclude  this  section,  it  is  not  the  analyst’s  job  to  select  an  alternative.  It  is  his 
job  to  make  recommendations  and  give  insights  based  on  the  data  received.  Using  the 
insights  and  recommendations  in  this  section,  the  NAIC  Commander  is  better  able  to 
choose  the  resource  allocation  alternative  that  is  best  suited  to  the  organization  based  on 
the  methodology  presented  in  Chapter  HI.  The  choice  is  left  to  the  decision  maker. 


4.4  Sensitivity  Analysis 

Logical  Decisions  analysis  software  was  used  to  perform  sensitivity  analysis  on  the 
NAIC  Commander’s  weights  [17],  Given  the  time  limitations  for  this  research,  sensitivity 
analysis  was  performed  only  on  the  weights  associated  with  the  Customer  Support  (0.6), 
Future  Requirements  (0.1),  and  Unit  Performance  (0.3)  values.  These  three  values  hold 
the  most  weight  overall.  To  show  this,  return  to  Figure  5  on  page  22.  The  overall  goal  is 
Commander’s  Value.  Under  this  is  the  Customer  Support,  Future  Requirements  and  Unit 
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Performance  weights  which  account  for  100%  of  the  Commander’s  Value.  Under  the 
Customer  Support  value  which  contains  60%  of  the  weight  for  the  overall  goal,  there  are 
the  Acquisition,  Operational,  and  Policymakers  values  with  weights  assigned.  At  most, 
the  Acquisition,  Operational,  and  Policymakers  weights  have  60%  of  the  overall  goal 
because  of  the  weight  assigned  to  the  Customer  Support  value.  Therefore,  if  the  proposal 
rankings  do  not  change  when  the  weights  assigned  to  the  Customer  Support,  Future 
Requirements,  and  Unit  Performance  values  are  changed  one  at  a  time,  then  the  rankings 
will  not  change  for  any  of  the  other  weights.  As  mentioned  in  Chapter  II,  the  weights 
remain  proportional  and  still  sum  to  one  when  sensitivity  analysis  is  performed.  To 
explain  this,  consider  the  initial  weights  given  to  the  top  three  values.  If  the  Customer 
Support  weight  was  changed  to  0.5,  then  the  remaining  two  weights  sum  to  0.5  and  retain 
their  original  proportions.  Future  Requirements  increases  to  0.125  and  Unit  Performance 
increases  to  0,375  which  is  still  a  proportional  weight  of  0.1  and  0.3  respectively, 
i.e.,0.25*0.5  =  0.125  and  0,75*0.5  =  0.375.  Sensitivity  analysis  was  performed  on  one 
weight  at  a  time  in  order  to  hold  that  one  weight  change  responsible  for  any  change  in 
rankings.  While  sensitivity  analysis  could  be  conducted  over  the  entire  range,  zero  to  one, 
of  these  weights,  the  sensitivity  analysis  used  here  varied  each  of  the  three  weights  by  0.1 
in  both  directions.  In  other  words.  Customer  Support  was  placed  at  0.5  and  0.7,  Future 
Requirements  at  0.0  and  0.2,  and  Unit  Performance  at  0.2  and  0.4.  The  sensitivity  analysis 
rankings  are  given  in  Appendix  K. 

Sensitivity  analysis  is  used  to  determine  how/if  things  change  in  response  to  one 
variable  being  changed  while  all  others  are  left  constant.  Appendix  K  has  six  graphs  that 
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show  how  the  rankings  changed  from  one  weight  change  to  the  next.  Figure  9  shows  the 
format  for  those  graphs.  While  the  next  ten  pages  could  be  used  to  show  each  and  every 
change  that  occurred  for  the  sensitivity  analysis  graphs,  only  general  insights  are  provided. 
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Figure  9.  Sensitivity  Analysis  Graph 


Impact  scores  for  the  most  part  were  less  when  the  Customer  Support  weight 
decreased  to  0.5  and  Unit  Performance  weight  increased  to  0.4.  Impact  scores  were 
higher  when  Customer  Support  weight  increased  to  0.7  and  Unit  Performance  weight 
decreased  to  0.2.  There  were  mixed  results  on  the  sensitivity  analysis  for  the  Future 
Requirements’  weight.  Decreasing  the  Future  Requirements  weight  to  zero,  DU2,  DU5, 
DUS,  and  DU9  received  lower  impact  scores,  and  DUl,  DU4,  and  DU6  received  higher 
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impact  scores.  Increasing  Future  Requirements  weight  to  0.2  yielded  lower  impact  scores 
for  DUl,  DU4,  and  DU6  and  higher  impact  scores  for  DU2,  DUS,  DU5,  and  DUS. 


4.5  NAIC  Implementation  Compared  to  Model  Results 

Table  3  provides  a  quick  comparison  between  the  model  results  and  the  NAIC 
selected  alternative.  Table  3  contains  each  proposal’s  ranking  based  on  its  impact  score 
from  Table  1  and  each  proposal’s  ranking  based  on  the  hnpact/Cost  ratio  from  Table  2. 

In  addition  to  these' two  rankings,  Table  3  shows  NAIC’s  selected  resource  allocation 
alternative  by  annotating  whether  each  proposal  was  cut  completely  (Y),  cut  partially  (P), 
or  not  cut  at  all  (N).  NAIC’s  cut  for  this  cycle  totaled  $8,544  million  in  external 
assistance. 

There  seems  to  be  little  correlation  between  what  NAIC  actually  chose  to  cut  and 
the  rank  ordering  from  the  two  methodologies.  Examining  the  ranking  of  the  proposals  to 
be  cut  provides  an  idea  of  the  amount  of  correlation.  From  Table  1,  the  first  22  proposals 
would  be  cut.  Adding  the  proposals  rankings  together  ,i.e.,  22+21+. .  .+1,  gives  a  total  of 
253.  Not  considering  the  proposals  that  NAIC  cut  partially,  the  sum  of  the  rankings  for 
the  proposals  they  cut  is  628.  This  amounts  to  approximately  a  2:5  ratio.  Using  Table  2, 
the  first  13  proposals  are  cut  resulting  in  a  value  of  91.  Based  on  Table  2  rankings,  the 
sum  for  NAIC’s  strategy  is  777  for  about  a  1:9  ratio. 
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Table  3.  Validation  and  Rank  Order 


Correlation  can  also  be  shown  through  the  amount  of  dollars  cut,  the  number  of 
proposals  cut  and  the  cumulative  impact  from  those  cut  proposals.  Based  on  impact 
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alone,  there  was  $2,626  million  of  the  $8,544  million  or  30.7%  of  the  actual  dollar  amount 
chosen  to  be  cut  which  represents  the  overlap  between  these  two  alternatives.  However, 
only  22  proposals  would  be  cut  if  the  impacts  were  used  compared  to  the  27  that  NAIC 
cut  partially  or  completely.  Remember  the  cumulative  impact  for  the  22  proposals  from 
Table  1  was  36.66.  Using  impact/cost,  there  was  $3,819  million  of  the  $8,544  million  or 
44.7%  of  the  actual  dollar  amount  chosen  to  be  cut  which  represents  the  overlap  between 
these  two  alternatives.  If  Table  2  were  used  to  determine  NAIC’s  resource  allocation 
strategy,  only  13  proposals,  compared  to  the  27  that  NAIC  selected,  would  be  cut  with  a 
cumulative  impact  of  23.21. 

The  cumulative  impact  scores  from  Table  1  and  Table  2  are  important.  The  total 
impact  score  for  NAIC’s  strategy  results  in  approximately  70.5  which  does  not  account 
for  any  of  the  proposals  that  were  partially  cut.  This  70.5  value  close  to  doubles  the 
impact  score  from  Table  1  and  is  over  three  times  as  large  as  the  impact  score  from  Table 
2.  In  other  words,  NAIC’s  impact  score  is  equivalent  to  cutting  approximately  $13,519 
million  in  Table  1  and  $17,507  million  in  Table  2. 

This  paragraph  highlights  some  of  the  possible  reasons  why  the  proposals 
recommended  for  cut  in  Table  1  and  Table  2  did  not  match  the  actual  proposals  cut  by 
NAIC.  To  begin,  the  SMT  decided  what  proposals  to  cut  with  some  input  from  their  DU 
Managers.  The  DU  Manager’s  scored  the  proposals.  The  percentages  may  have  been 
higher  if  the  SMT  had  scored  the  proposals.  As  an  example,  DU8  and  DU9  have  different 
DU  Managers  but  the  same  representative  at  the  SMT.  DU8’s  proposals  had  an  average 
impact  score  of  4.53  and  DU9’s  proposals  had  an  average  impact  score  of  1.96. 
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However,  more  money  was  cut  from  DUS  than  DU9.  Another  possibility  is  that  the 
commander’s  weights  were  not  accurate  for  this  cycle.  Sensitivity  analysis  can  be  used  to 
zero  in  on  the  weights  that  would  more  adequately  reflect  how  this  cycle’s  proposals  were 
selected.  A  key  place  to  start  is  with  Future  Requirements  because  this  seemed  to  be  the 
only  weight  that  discriminated  between  DUs.  Also,  the  scoring  sheets  may  not  have 
accurately  reflected  the  intended  measures,  or  there  may  be  additional  measures  and  values 
within  the  value  hierarchy  that  were  not  exposed.  These  reasons  and  many  more  would 
cause  the  methodology  to  not  accurately  reflect  the  NAIC  Commander’s  resource 
allocation  strategy. 
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V.  Conclusions  and  Recommendations 


5.1  Conclusions 

The  objective  of  this  research  was  to  provide  NAIC  with  a  tool  or  method  for 
resource  allocation  that  would  save  time,  generate  better  alternatives,  and  select  an 
alternative  based  upon  the  NAIC  Commander’s  values  and  preferences.  The  study 
concludes  that  decision  analysis  using  value-focused  thinking  provided  a  method  for 
accomplishing  this  objective.  The  method  took  less  time  to  develop  alternatives  and  can 
be  used  to  select  an  alternative  based  upon  the  NAIC  Commander’s  values  and 
preferences.  Whether  the  alternatives  are  better  has  not  been  determined. 

Of  key  importance  throughout  this  whole  research  effort  is  the  attainment  of  the 
NAIC  Commander’s  values  and  preferences  which  will  be  used  to  help  focus  and  direct 
NAIC.  Whether  these  values  and  preferences  are  used  to  select  a  resource  allocation 
strategy,  new  program  implementation,  customer  support,  or  how  NAIC  does  business,  all 
are  candidates  for  application  of  the  NAIC  Commander’s  values  and  preferences.  NAIC 
will  be  better  focused  and  have  a  better  understanding  of  what  their  conomander  values 
and  how  he  prefers  those  values  through  this  research. 

Also  of  use  to  NAIC  was  the  scoring  mechanism  used  for  this  methodology. 

Using  a  line  scale  allows  them  to  obtain  more  accurate  and  timely  information  from  both 
internal  and  external  customers  for  the  metrics  that  organizations  are  now  being  required 
to  keep. 
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5.2  Lessons  Learned 


Resource  allocation  is  a  very  dynamic  process  and  may  be  changed  up  to  the  last 
moment.  Having  a  method  that  can  accommodate  this  active  process  no  matter  the 
change  is  critical,  as  was  learned  in  the  final  weeks  of  this  research. 

Saving  time  or  taking  short  cuts  in  a  process  may  not  always  be  good.  For  this 
cycle  at  NAIC,  only  28  of  the  41  measures  were  used  to  score  and  rank  the  proposals. 
Even  if  the  remaining  13  measures  seemed  to  have  no  impact,  they  should  have  been 
included  anyway.  Surprises  can  always  occur  and  may  tip  the  scales  between  one 
alternative  and  another. 

Of  big  concern  to  NAIC  was  the  question  of  subjectivity  and  how  to  deal  with  it. 
Are  the  DU  Managers  intentionally  inflating  the  impact  that  a  dollar  amount  cut  would 
have  on  them  or  are  they  being  honest?  If  they  are  honest,  are  they  more  likely  to  get  their 
resources  cut  because  they  do  not  score  as  high  as  their  peers  who  inflated  their  scores? 
These  issues  abound  everywhere  it  seems.  NAIC  dealt  with  these  issues,  as  most 
organizations  do,  by  not  completely  trusting  the  impact  given  by  the  DU  Managers.  It 
may  have  been  more  appropriate  to  have  had  the  SMT  score  the  proposals  or  at  least 
review  them. 
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5.3  Future  Research 


There  are  many  techniques  and  methods  that  perform  resource  allocation. 
Additional  research  should  be  conducted  to  determine  if  a  better  method  than  the  one  used 
is  available.  In  addition  to  the  NAIC  Commander’s  values  and  weights,  it  would  be 
appropriate  and  useful  to  obtain  the  values  and  weights  held  by  each  member  of  the  SMT. 
While  this  is  a  vast  undertaking,  the  values  and  weights  held  by  the  SMT  could  prove  very 
valuable  to  NAIC. 


5.4  Recommendations 

While  this  methodology  scored  proposals  that  were  used  to  develop  the 
alternatives  for  NAIC’s  resource  allocation  strategy,  the  validation  of  this  methodology 
showed  a  disconnect.  That  difference  could  have  occurred  in  many  areas.  Using 
sensitivity  analysis,  it  may  be  possible  to  account  for  this  difference  in  the  commander’s 
weights.  As  already  mentioned,  the  place  to  start  is  with  Future  Requirements.  LDW 
performs  sensitivity  analysis  very  well,  but  provides  no  ability  to  perform  cost-benefit 
analysis  which  produced  better  results  than  those  based  on  the  impact  alone.  A  different 
software  package  may  do  a  better  job  and  should  be  explored. 

The  anomaly  may  also  have  been  in  the  SMT’s  recommendations.  An  alternative 
may  be  to  have  the  SMT  review  the  scoring  sheets  before  the  analyst  scores  them  or  to 
shift  the  burden  of  resource  allocation  off  of  the  DU  Managers  and  onto  the  2-Letter 
Directorate  Chiefs  who  compose  the  SMT.  This  would  make  the  2-Letters  responsible  for 
filhng  out  the  scoring  sheets  and  ensuring  the  scoring  sheets  adequately  convey  the 
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measure  being  scored.  Also,  regardless  of  whether  or  not  a  measure  might  not  apply  to  a 
situation,  score  that  measure  anyway.  Surprises  may  provide  a  much  better  ranking  of  the 
alternatives  or  proposals  than  those  arrived  at  by  leaving  measures  out  of  the  scoring 
process. 

Lastly,  the  value  hierarchy  should  be  examined  and  revisited  to  ensure  no  values  or 
measures  were  missed.  The  value  hierarchy,  along  with  the  weights,  should  be  reviewed 
annually  or  as  needed  to  maintain  their  applicability  and  usefulness. 
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Appendix  A.  NAIC  Commander's  Value  Hierarchy 
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Appendix  B.  Scoring  Sheets 


INSTRUCTIONS 


All  questions  are  based  on  resource  allocation  cuts  in  contracts,  3400,  for  FY99. 
When  gauging  the  impact  of  a  contract,  you  should  ask  yourself,  "How  will  this  contract 
cut  affect  NAIC  in  this  area?"  Also,  your  ratings  should  NOT  reflect  where  you  are  now. 
For  example,  NAIC  may  already  be  two  weeks  late  getting  products  to  the  Acquisition 
Commimity  and  this  contract  cut  will  make  your  products  three  weeks  late.  This  measure 
would  be  rated  as  only  one  week  late  because  the  contract  cut  is  responsible  for  making 
the  products  one  week  late,  not  three.  Rating  the  measures  in  this  way  will  provide  a 
consistent  basehne. 

Score  the  questions  using  the  scale  provided  for  each  measure.  Place  an  X  on  the 
scale  reflecting  the  impact  of  this  measure  on  your  DU  if  this  contract  were  to  be  cut  in  its 
entirety.  This  X  will  later  be  translated  into  a  numeric  score.  If  this  contract  has  no 
impact  or  is  not  applicable  for  your  DU  on  any  given  measure,  then  place  your  X  on  the 
far  left  hand  side  of  the  scale  (above  No  Impact). 

The  designator  at  the  top  of  each  page  keeps  the  actual  scoring  sheets  unclassified 
when  they  are  collected.  Please  do  not  write  the  name  of  the  contract  on  any  of  the 
following  pages. 
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Appendix  B.  Scoring  Sheets 


CUSTOMER  SUPPORT 

Designator _ 

Acquisition:  Capability 

If  this  contract  is  cut,  how  will  it  affect  NAIC’s  ability  to  fulfill  DoDIPP 
requirements  for  the  Acquisition  Community? 


No  Impact 


Lane 

Lost 


Acquisition:  Capacity 

If  this  contract  is  cut,  how  will  it  reduce  NAIC’s  ability  to  satisfy 
acquisition  customer  requirements?  In  other  words,  will  your  DU  still  have 
the  ability  to  satisfy,  fulfill,  and  accommodate  Acquisition  customer 
requirements  that  are  placed  on  your  DU? 


No  Impact 


100% 

Reduction 


Acquisition:  Quality  .Accessibility 

If  this  contract  is  cut,  how  will  it  affect  NAIC’s  ability  to  provide  an 
accessible  product  to  the  Acquisition  Community?  Inaccessible  can  be 
defined  in  terms  of  the  medium,  such  as  CD-ROM,  Network  linkups,  etc., 
or  classification  level. 


No  Impact 


Inaccessible  Inaccessible 

by  Medium  Ijy  Classification 


Acquisition:  Quality: Accuracy 

If  this  contract  is  cut,  how  will  it  affect  the  accuracy  of  products  provided 
to  the  Acquisition  Community?  With  accuracy,  think  in  terms  of  the 
information  provided.  Will  the  information  provided  be  any  less  accurate 
because  of  this  cut? 


No  Impact 


Slightly 

Inaccurate 


Inaccurate 


Appendix  B.  Scoring  Sheets 


CUSTOMER  SUPPORT 

Designator _ 

Acquisition:  Quality  .'Relevance 

If  this  contract  is  cut,  how  will  it  impact  the  ability  to  provide  a  relevant 
and  tailored  product  to  the  Acquisition  Community? 


No  Impact 


Not  Tailored 


Not  Relevant 


Acquisition:  Quality :Timeliness 

If  this  contract  is  cut,  how  will  it  impact  the  ability  to  provide  an  on-time 
product  to  the  Acquisition  Community?  Notice  that  the  timeline  on  this 
scale  is  different  from  the  timeline  of  Operational  and  Policymakers 
Communities.  Do  not  mark  this  scale  on  how  late  your  products  are  now, 
but  on  how  much  later  your  products  will  become  if  this  contract  is  cut. 


No  Impact 


Over  4 
Weeks  Late 


Operational:  Capability 

If  this  contract  is  cut,  how  will  it  affect  your  ability  to  fulfill  DoDIPP 
requirements  for  the  Operational  Community? 


No  Impact 


Lane 

Lost 


Operational:  Capacity 

If  this  contract  is  cut,  how  will  it  reduce  NAIC’s  ability  to  satisfy 
operational  customer  requirements?  In  other  words,  will  your  DU  still 
have  the  ability  to  satisfy,  fulfill,  and  accommodate  Operational  customer 
requirements  that  are  placed  on  your  DU? 


No  Impact 


100% 

Reduction 
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CUSTOMER  SUPPORT 

Designator _ 

Operational:  Quality :Accessibility 

If  this  contract  is  cut,  how  will  it  affect  your  ability  to  provide  an  accessible 
product  to  the  Operational  Community?  Inaccessible  can  be  defined  in 
terms  of  the  medium,  such  as  CD-ROM,  Network  linkups,  etc.,  or 
classification  level. 


No  Impact 


Inaccessible  Inaccessible 

by  Medium  by  Classification 


Operational:  Quality: Accuracy 

If  this  contract  is  cut,  how  will  it  affect  the  accuracy  of  products  provided 
to  the  Operational  Community?  With  accuracy,  think  in  terms  of  the 
information  provided.  Will  the  information  provided  be  any  less  accurate 
because  of  this  cut? 


No  Impact  Slightly  Inaccurate 

Inaccurate 


Operational:  Quality  .Relevance 

If  this  contract  is  cut,  how  will  it  impact  the  ability  to  provide  a  relevant 
and  tailored  product  to  the  Operational  Community? 


No  Impact 


Not  Tailored 


Not  Relevant 


Operational:  Quality:! imeliness 

If  this  contract  is  cut,  how  will  it  impact  the  ability  to  provide  an  on-time 
product  to  the  Operational  Community?  Do  not  mark  this  scale  on  how 
late  your  products  are  now,  but  on  how  much  later  your  products  will 
become  if  this  contract  is  cut. 


No  Impact 


Over  2 
Weeks  Late 
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CUSTOMER  SUPPORT 

Designator _ 

Policymakers:  Capability 

If  this  contract  is  cut,  how  will  it  affect  your  ability  to  fulfill  DoDIPP 
requirements  for  the  Policymakers  Community? 


No  Impact 


Lane 

Lost 


Policymakers:  Capacity 

If  this  contract  is  cut,  how  will  it  reduce  NAIC’s  ability  to  satisfy 
Policymakers  customer  requirements?  In  other  words,  will  your  DU  still 
have  the  ability  to  satisfy,  fulfill,  and  accommodate  Policymakers  customer 
requirements  that  are  placed  on  your  DU? 


No  Impact 


100% 

Reduction 


Policymakers:  Quality  .Accessibility 

If  this  contract  is  cut,  how  will  it  affect  NAIC’s  ability  to  provide  an 
accessible  product  to  the  Policymakers  Community?  Inaccessible  can  be 
defined  in  terms  of  the  medium,  such  as  CD-ROM,  Network  linkups,  etc., 
or  classification  level. 


No  Impact  Inaccessible 

by  Medium 


Inaccessible 
by  Classification 


Policymakers:  Quality: Accuracy 

If  this  contract  is  cut,  how  will  it  affect  the  accuracy  of  products  provided 
to  the  Policymakers  Community?  With  accuracy,  think  in  terms  of  the 
information  provided.  Will  the  information  provided  be  any  less  accurate 
because  of  this  cut? 


No  Impact 


Slightly  Inaccurate 

Inaccurate 
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CUSTOMER  SUPPORT 

Designator _ 

Policymakers :  Quality  ‘.Relevance 

If  this  contract  is  cut,  how  will  it  impact  NAIC’s  ability  to  provide  a 
relevant  and  tailored  product  to  the  Policymakers  Community? 


No  Impact 


Not  Tailored 


Not  Relevant 


Policymakers:  Quality :Timeliness 

If  this  contract  is  cut,  how  will  it  impact  NAIC’s  ability  to  provide  an  on- 
time  product  to  the  Policymakers  Community?  Do  not  mark  this  scale  on 
how  late  your  products  are  now,  but  on  how  much  later  your  products  will 
become  if  this  contract  is  cut. 


No  Impact 


Over  2 
Weeks  Late 


Appendix  B.  Scoring  Sheets 


FUTURE  REQUIREMENTS 

Designator _ 

Futures  Analysis 

If  this  contract  is  cut,  how  will  it  affect  NAIC’s  ability  to  perform  futures 
analysis?  Base  your  response  upon  the  five  factors  listed  in  the  Master 
Plan:  Economics,  Leadership  Personality,  Military  Doctrine, 
Inter/Intrastate  Relationships,  and  Technology.  At  the  100%  level,  all 
ability  to  perform  futures  analysis  is  lost. 


No  Impact  100% 

Loss 


Integrated  Air  Defense  System 

If  this  contract  is  cut,  how  will  it  affect  NAIC’s  ability  to  produce  a  threat 
characterization  of  an  enemy  IADS  in  terms  of  its  strategy,  doctrine, 
tactics,  training,  maintenance  limitations,  and  technical  characteristics? 


100% 
Loss 

Information  Operations:  Defensive 

If  this  contract  is  cut,  how  will  it  affect  NAIC's  ability  to  perform  counter 
denial/deception? 


No  Impact 


No  Impact 


Cannot 

Perform 


Information  Operations:  Defensive 

If  this  contract  is  cut,  how  will  it  affect  NAIC's  ability  to  produce 
intelligence  about  foreign  countries’  capabilities  or  intentions  to  conduct 
information  warfare? 


No  Impact  100% 

Loss 
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FUTURE  REQUIREMENTS 

Designator _ 

Information  Operations:  Offensive 

If  this  contract  is  cut,  how  will  it  affect  NAIC's  ability  to  assess  or  produce 
foreign  countries’  capabilities  and  intentions  associated  with  command, 
control,  communications,  and  computers.  Base  your  response  on  three 
areas:  (1)  Systematically  examining  aU  possible  explanations  for  events,  (2) 
understanding  the  assumptions  that  are  critical  to  the  assessment,  and  (3) 
identifying  the  types  of  new  information  or  changes  in  events  that  would 
cause  us  to  change  the  assessment? 


No  Impact 


3  Areas 
Cut 


Information  Operations:  Offensive 

If  this  contract  is  cut,  how  will  it  affect  NAIC's  ability  to  assess  or  produce 
foreign  countries’  capabilities  and  intentions  associated  with  electronic 
systems.  Base  your  response  on  three  areas:  (1)  Systematically  examining 
all  possible  explanations  forevents,  (2)  understanding  the  assumptions  that 
are  critical  to  the  assessment,  and  (3)  identifying  the  types  of  new 
information  or  changes  in  events  that  would  cause  us  to  change  the 
assessment? 


No  Impact 


3  Areas 
Cut 


Information  Operations:  Offensive 

If  this  contract  is  cut,  how  will  it  affect  NAIC's  ability  to  assess  or  produce 
foreign  countries’  capabilities  and  intentions  associated  with  space  systems. 
Base  your  response  on  three  areas:  (1)  Systematically  examining  all 
possible  explanations  for  events,  (2)  understanding  the  assumptions  that  are 
critical  to  the  assessment,  and  (3)  identifying  the  types  of  new  information 
or  changes  in  events  that  would  cause  us  to  change  the  assessment? 


No  Impact 


3  Areas 
Cut 


Appendix  B.  Scoring  Sheets 


UNIT  PERFORMANCE 

Designator _ 

Proliferation 

If  this  contract  is  cut,  how  will  it  decrease  NAIC’s  understanding  the  ability 
of  countries  to  assimilate  procured  or  indigenously  developed  technology 
into  weapon  systems’  development  and  employment? 


No  Impact 


100% 

Decrease 


People  :Flexibility 

If  this  contract  is  cut,  how  will  it  affect  the  flexibility  of  the  people  within 
your  DU?  Your  DU  may  already  have  no  flexibility.  However,  suppose 
your  DU  had  a  great  deal  of  flexibility  and  now  this  contract  cut  occurs. 
How  will  it  affect  your  DU? 


No  Impact 


No  Flexibility 


People  :Promotion  Frontier 

If  this  contract  is  cut,  how  will  it  affect  the  ability  of  your  personnel  to  get 
promoted?  In  this  case,  consider  how  this  cut  will  affect  your  personnel. 
WiU  they  be  performing  tasks  and  jobs  that  would  take  them  away  from 
and  not  contribute  to  doing  the  things  that  are  important  for  promotion? 


No  Impact 


Forced 

Retirement 


People:! raining 

If  this  contract  is  cut,  how  will  it  affect  the  amount  of  training  your 
personnel  receive?  Your  DU  may  already  not  get  the  training  it  needs. 
Base  this  rating  not  on  where  you  are  now,  but  on  how  this  contract  cut 
will  prevent  your  personnel  from  getting  the  training  they  need. 


No  Impact 


No  Training 
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UNIT  PERFORMANCE 

Designator _ 

People :  Leadership  .'Communication 

If  this  contract  is  cut,  how  will  it  affect  your  DU's  ability  to  communicate 
with  organizations  internal  to  NAIC?  Base  your  response  upon  the  time 
available  for  communication,  quality  of  communication  (clear,  concise, 
accurate),  and  communicating  with  the  right  person  or  group. 


No  Impact 


No 

Communication 


People:  Leadership  .Community  Work 

If  this  contract  is  cut,  how  will  it  affect  the  ability  of  members  in  your  DU 
to  perform  community  work?  Base  your  response  on  the  amount  of  time 
available  for  your  personnel  to  perform  community  work.  Not  to  be  a 
broken  record,  but  remember  this  rating  is  not  focused  on  where  your  DU 
is  now.  Focus  on  the  contract  cut  and  how  it  affects  this  measure. 


No  Impact 


No  Community 
Work  Possible 


People:  Quality  ofLife:TDY  Requirements 

How  will  this  contract  cut  increase  the  TDY  requirements  for  people  in 
your  DU?  Will  they  be  required  to  go  TDY  more  because  requirements 
have  increased  while  manpower  decreased  or  stayed  the  same? 


No  Impact 


TDY  Greatly 
Increased 


People:  Quality  ofLife:Workload 

How  will  this  contract  add  to  the  workload  of  your  employees?  This 
assumes  that  if  your  DU  personnel  are  overworked,  there  is  very  little  if 
any  time  for  communication  and  community  work.  This  scale  is  based  on 
hours  per  week  extra  required  to  fulfill  mission  requirements. 


No  Impact 


15 
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UNIT  PERFORMANCE 

Designator _ 

People:  Quality  ofLife:Work  Environment 

How  does  this  contract  cut  affect  your  work  environment?  Is  the  stress 
level  preventing  work  from  being  accomplished?  Does  air  quality,  poor 
ventilation,  desk/chair  quality  and/or  placement  hinder  work  being 
accomplished? 


No  Impact 


Work  is 

Seriously  Hindered 


Processes ‘.Planning 

If  this  contract  is  cut,  how  will  it  affect  your  DU's  ability  to  produce  plans 
and/or  respond  to  plans?  Producing  plans  pertains  to  methods  for 
collecting  metrics,  providing  continuity  folders,  and  any  other  type  of  plan 
that  would  organize  or  run  your  DU  more  efficiently.  Responding  to  plans 
is  what  you  are  doing  now  through  this  scoring  sheet  and  may  be  external 
or  internal  for  your  DU. 


No  Impact 


Greatly  Impacts 
Ability  to  Plan 


Processes  .‘Safety 

If  this  contract  is  cut,  how  will  it  affect  yoiu’  DU's  ability  to  adhere  to 
safety  standards?  This  rating  does  not  distinguish  between  tripping  over  a 
power  cord,  producing  an  environmental  hazard,  and  other  safety  hazards. 
It  applies  only  to  an  inability  to  adhere  to  safety  standards.  An  inability  to 
adhere  is  also  different  from  attention  to  detail. 


No  Impact  Unable  to  Adhere 

to  Safety  Standards 


Processes. ‘Security 

If  this  contract  is  cut,  how  will  it  affect  the  possibility  of  compromising 
information  for  your  DU?  Consider  information,  physical,  and  personnel 
security  along  with  OPSEC,  COMPUSEC,  COMSEC,  EMSEC,  etc.  when 
rating  this  measure. 


No  Impact 


Chances  of  Compromise 
Slightly  Increased 


Compromise 
Highly  Likely 
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UNIT  PERFORMANCE 

Designator _ 

Processes:  Data.Management 

If  this  contract  is  cut,  how  will  it  affect  your  DU's  ability  to  manage  data? 
Is  there  too  much  data  to  manage,  technicians  unavailable  to  manage  the 
data,  or  data  management  software  unavailable? 


No  Impact 


100% 

Cut 


Process:  Resources:  Equipment 

If  this  contract  is  cut,  how  will  it  affect  your  DU's  equipment  resources? 
Will  the  equipment  collect  dust  because  it  is  outdated,  no  longer  necessary 
or  no  one  understands  how  to  use  it? 


No  Impact 


Equipment 

Unused 


Process:  Resources  .Facility 

If  this  contract  is  cut,  how  will  it  affect  your  DU's  facility  resources? 
Consider  whether  facility  resources  are  being  used  inefficiently  or  not  at  all 
and  whether  upgrades  are  needed  to  complete  a  job? 


No  Impact 


100% 

Resource  Lost 


Processes:  Data.Analysis 

If  this  contract  is  cut,  how  will  it  affect  your  DU's  ability  to  analyze  data? 
For  this  scale  consider  the  accuracy,  reliability,  and  accessibility  of  the  data. 
Would  data  no  longer  be  available  or  would  data  analysis  no  longer  be 
possible.? 


No  Impact 


100% 

Cut 


Appendix  C.  Scoring  Functions  at  DU  Level 


Decision  Unit  Scoring  Functions 


o 

u 


Use  Scales  from  Scoring  Sheet  for  X-axis 
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Information  Exploitation  Attributes 

INFOEX[0][INFO_EX],INFOEX[1]  [INFO_EX]  ,INFOEX[2]  [INFO_EX],INFOEX[3]  [I 
NFO_EX]  ,INFOEX[4]  [INFO_EX]  ,INFOEX[5]  [INFO_EX],INFOEX[6]  [INFO_EX]  ,INF 
OEX[7]  [INFO_EX]  ,INFOEX[8]  [INFO_EX]  ,INFOEX[9]  [INFO_EX]  ,INFOEX[  10]  [INF 
0_EX],INF0EX[11][INF0_EX],INF0EX[12][INF0_EX],INF0EX[13][INF0_EX]JN 
FOEX[14][INFO_EX],INFOEX[15][INFO_EX],INFOEX[16][INFO_EX],INFOEX[17][ 
INFO_EX],INFOEX[18][INFO_EX],INFOEX[19][INFO_EX],INFOEX[20][INFO_EX] 
,INFOEX[21][INFO_EX],INFOEX[22][INFO_EX],INFOEX[23][INFO_EX],INFOEX[ 
24]  [INFO_EX]  ,INFOEX[25]  [INFO_EX]  ,INFOEX[26]  [INFO_EX]  ,INFOEX[27]  [INFO_ 
EX],INFOEX[28][INFO_EX],INFOEX[29][INFO_EX]JNFOEX[30][INFO_EX],INFO 
EX[3 1]  [INFO_EX],INFOEX[32][INFO_EX],INFOEX[33]  [INFO_EX]  ,INFOEX[34][IN 
FO_EX]  ,INFOEX[35]  [INFO_EX]  ,INFOEX[36]  [INFO_EX]  ,INFOEX[37]  [INFO_EX]  ,I 
NFOEX[38][INFO_EX],INFOEX[39][INFO_EX],INFOEX[40][INFO_EX],INFOEX[41 
]  [INFO_EX]  ,INFOEX[42]  [INFO_EX],INFOEX[43]  [INFO_EX] 

MASINT  Attributes: 

MASIN[0][MASINT],MASIN[1][MASINT],MASIN[2][MASINT],MASIN[3][MASINT 

],MASIN[4][MASINT],MASIN[5][MASINT],MASIN[6][MASINT],MASIN[7][MASIN 

T],MASIN[8][MASINT],MASIN[9][MASINT],MASIN[10][MASINT],MASIN[11][MA 

SINT],MASIN[12][MASINT],MASIN[13][MASINT],MASIN[14][MASINT],MASIN[1 

5][MASINT],MASIN[16]  [MASINT], MASIN[17][MASINT],MASIN[18][MASINT],M 

ASIN[  19]  [MASINT], MASIN[20]  [MASINT|,MASIN[2 1  ]  [MASINT]  ,MASIN[22]  [MASI 

NT],MASIN[23][MASINT],MASIN[24][MASINT],MASIN[25][MASINT],MASIN[26] 

[MASINT],MASIN[27][MASINT],MASIN[28][MASINT],MASIN[29][MASINT],MAS 

IN[30][MASINT],MASIN[31][MASINT],MASIN[32][MASINT],MASIN[33][MASINT 

],MASIN[34][MASINT],MASIN[35][MASINT],MASIN[36][MASINT],MASIN[37][M 

ASINT],MASIN[38][MASINT],MASIN[39][MASINT],MASIN[40][MASINT],MASIN[ 

41][MASINT|,MASIN[42][MASINT]-INFOEX[42][INFO_EX],MASIN[43][MASINT] 

C4I  Attributes: 

CFOURI[0][C4I],CFOURI[l][C4I],CFOURI[2][C4r],CFOURI[3][C4I],CFOURI[4][C4I 

]  ,CFOURI[5][C4I]  ,CFOURI[6]  [C4I],CFOURI[7]  [C4I]  ,CFOURI[8]  [C4I]  ,CFOURI[9]  [C4 

I],CFOURI[10][C4I],CFOURI[11][C4I],CFOURI[12][C4I],CFOURI[13][C4I],CFOURI[ 

14][C4I],CFOURI[15][C4I],CFOURI[16][C4I],CFOURI[17][C4I],CFOURI[18][C4I],C 

FOURI[19][C4I],CFOURI[20][C4I],CFOURI[21][C4I],CFOURI[22][C4I],CFOURI[23] 

[C4I],CFOURI[24][C4I],CFOURI[25][C4I],CFOURI[26][C4I],CFOURI[27][C4I],CFO 

URI[28][C4I],CFOURI[29][C4I],CFOURI[30][C4I],CFOURI[31][C4I],CFOURI[32][C4 

I],CFOURI[33][C4I],CFOURI[34][C4I],CFOURI[35][C4I],CFOURI[36][C4I],CFOURI[ 

37][C4I],CFOURI[38][C4I],CFOURI[39][C4I],CFOURI[40][C4I],CFOURI[41][C4I],C 

FOURI[42][C4I]-MASIN[42][MASINT],CFOURI[43][C4I] 
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Electronic  Systems  Attributes: 

ELEC[0][ELEC_SYS],ELEC[1][ELEC_SYS],ELEC[2][ELEC_SYS],ELEC[3][ELEC_S 

YS]  ,ELEC[4]  [ELEC_S  YS]  ,ELEC[5]  [ELEC_S  YS]  ,ELEC[6]  [ELECTS  YS]  ,ELEC[7]  [EL 

EC_S  YS]  ,ELEC[8]  [ELEC_S  YS],ELEC[9]  [ELEC_S  YS]  ,ELEC[  10]  [ELEC_S  YS],ELEC[ 

11][ELEC_SYS],ELEC[12][ELEC_SYS],ELEC[13][ELEC_SYS],ELEC[14][ELEC_SY 

S],ELEC[15][ELEC_SYS],ELEC[16][ELEC_SYS],ELEC[17][ELEC_SYS],ELEC[18][ 

ELEC_SYS],ELEC[19][ELEC_SYS],ELEC[20][ELEC_SYS],ELEC[21][ELEC_SYS],E 

LEC[22][ELEC_SYS],ELEC[23][ELEC_SYS],ELEC[24][ELEC_SYS],ELEC[25][ELE 

C_SYS],ELEC[26][ELEC_SYS],ELEC[27][ELEC_SYS],ELEC[28][ELEC_SYS],ELEC 

[29]  [ELEC_S  YS]  ,ELEC[30][ELEC_S  YS],ELEC[3 1]  [ELEC_S  YS],ELEC[32]  [ELEC_S 

YS],ELEC[33][ELEC_SYS],ELEC[34][ELEC_SYS],ELEC[35][ELEC_SYS],ELEC[36] 

[ELEC_SYS],ELEC[37][ELEC_SYS],ELEC[38][ELEC_SYS],ELEC[39][ELEC_SYS], 

ELEC[40][ELEC_SYS]JELEC[41][ELEC_SYS],ELEC[42][ELEC_SYS]- 

CFOURI[42]  [C4I]  ,ELEC[43]  [ELEC_S  YS] 

Aerodynamic  Systems  Attributes: 

AERO[0][AERO_SYS],AERO[1][AERO_SYS],AERO[2][AERO_SYS],AERO[3][AER 

0_SYS],AER0[4][AER0_SYS],AER0[5][AER0_SYS],AER0[6][AER0_SYS],AER0 

[7][AERO_SYS],AERO[8][AERO_SYS],AERO[9][AERO_SYS],AERO[10][AERO_S 

YS],AER0[11][AER0_SYS],AER0[12][AER0_SYS],AER0[13][AER0_SYS],AER0[ 

14]  [AER0_S  YS] ,  AER0[  15]  [AERO_S  YS],AERO[  1 6]  [AERO_S  YS] ,  AERO[  17]  [AERO_ 

SYS]  ,AERO[  1 8]  [AERO_S  YS]  ,AERO[  19][AERO_S  YS]  ,AERO[20]  [AERO_S  YS]  ,AER 

0[21][AER0_SYS],AER0[22][AER0_SYS],AER0[23][AER0_SYS],AER0[24][AER 

0_SYS],AER0[25][AER0_SYS],AER0[26][AER0_SYS],AER0[27][AER0_SYS],A 

ERO[28][AERO_SYS],AERO[29][AERO_SYS],AERO[30][AERO_SYS],AERO[31][A 

ER0_SYS],AER0[32][AER0_SYS],AER0[33][AER0_SYS],AER0[34][AER0_SYS] 

,AERO[35]  [AERO_S  YS],AERO[36]  [AERO_SYS],  AERO[37]  [AERO_S  YS],AERO[38] 

[AERO_SYS],AERO[39][AERO_SYS],AERO[40][AERO_SYS],AERO[41][AERO_SY 

S] ,AERO[42][AERO_SYS]-ELEC[42][ELEC_SYS],AERO[43][AERO_SYS] 

SIGINT  Attributes: 

SIGIN[0][SIGINT],SIGIN[1][SIGINT],SIGIN[2][SIGINT],SIGIN[3][SIGINT],SIGIN[4 

][SIGINT],SIGIN[5][SIGINT],SIGIN[6][SIGINT],SIGIN[7][SIGINT],SIGIN[8][SIGIN 

T] ,SIGIN[9]  [SIGINT]  ,SIGIN[  10]  [SIGINT], SIGIN[  1 1]  [SIGINT]  ,SIGIN[  12]  [SIGINT],S 
IGIN[13][SIGINT],SIGIN[14][SIGINT],SIGIN[15][SIGINT],SIGIN[16][SIGINT],SIGI 
N[17][SIGINT],SIGIN[18][SIGINT],SIGIN[19][SIGINT],SIGIN[20][SIGINT],SIGIN[2 
1][SIGINT],SIGIN[22][SIGINT],SIGIN[23][SIGINT],SIGIN[24][SIGINT],SIGIN[25][ 
SIGINT], SIGIN[26][SIGINT],SIGIN[27][SIGINT],SIGIN[28][SIGINT],SIGIN[29][SI 
GINT]  ,SIGIN[30][SIGINT]  ,SIGIN[3 1][SIGINT],SIGIN[32]  [SIGINT]  ,SIGIN[33]  [SIGI 
NT],SIGIN[34][SIGINT],SIGIN[35][SIGINT],SIGIN[36][SIGINT],SIGIN[37][SIGINT 
]  ,SIGIN[38]  [SIGINT]  ,SIGIN[39][SIGINT],SIGIN[40]  [SIGINT]  ,SIGIN[4 1]  [SIGINT]  ,SI 
GIN[42][SIGINT]-AERO[42][AERO_SYS],SIGIN[43][SIGINT] 


91 


Appendix  F.  Attributes 


Global  Threat  Attributes: 

GLOBA[0][GLOBAL],GLOBA[1][GLOBAL],GLOBA[2][GLOBAL],GLOBA[3][GLO 

BAL],GLOBA[4][GLOBAL],GLOBA[5][GLOBAL],GLOBA[6][GLOBAL],GLOBA[7] 

[GLOBAL]  ,GLOB  A[8]  [GLOBAL]  ,GLOBA[9][GLOB  AL]  ,GLOB  A[  10]  [GLOBAL]  ,GL 

OBA[ll][GLOBAL],GLOBA[12][GLOBAL],GLOBA[13][GLOBAL],GLOBA[14][GL 

OBAL],GLOBA[15][GLOBAL],GLOBA[16][GLOBAL],GLOBA[17][GLOBAL],GLO 

BA[18][GLOBAL],GLOBA[19][GLOBAL],GLOBA[20][GLOBAL],GLOBA[21][GLO 

BAL],GLOBA[22][GLOBAL],GLOBA[23][GLOBAL],GLOBA[24][GLOBAL],GLOB 

A[25][GLOBAL],GLOBA[26][GLOBAL],GLOBA[27][GLOBAL],GLOBA[28][GLOB 

AL]  ,GLOB  A[29]  [GLOBAL]  ,GLOB  A[30]  [GLOB  AL],GLOB  A[3 1  ]  [GLOBAL]  ,GLOB  A[ 

32][GLOBAL],GLOBA[33][GLOBAL],GLOBA[34][GLOBAL],GLOBA[35][GLOBAL 

],GLOBA[36][GLOBAL],GLOBA[37][GLOBAL],GLOBA[38][GLOBAL],GLOBA[39] 

[GLOBAL], GLOBA[40][GLOBAL],GLOBA[41][GLOBAL],GLOBA[42][GLOBAL]- 

SIGIN[42]  [SIGINT]  ,GLOB  A[43]  [GLOBAL] 

Engineering  Systems  Attributes: 

ENGI[0]  [ENGINEER], ENGI[  1]  [ENGINEER]  ,ENGI[2]  [ENGINEER]  ,ENGI[3]  [ENGIN 

EER],ENGI[4][ENGINEER],ENGI[5][ENGINEER],ENGI[6][ENGINEER],ENGI[7][E 

NGINEER],ENGI[8][ENGINEER],ENGI[9][ENGINEER],ENGI[10][ENGINEER],EN 

GI[11][ENGINEER],ENGI[12][ENGINEER],ENGI[13][ENGINEER],ENGI[14][ENGI 

NEER],ENGI[15][ENGINEER],ENGI[16][ENGINEER],ENGI[17][ENGINEER],ENGI 

[18]  [ENGINEER],ENGI[  19]  [ENGINEER], ENGI[20]  [ENGINEER], ENGI[21]  [ENGINE 

ER],ENGI[22][ENGINEER],ENGI[23][ENGINEER],ENGI[24][ENGINEER],ENGI[25 

][ENGINEER],ENGI[26][ENGINEER],ENGI[27][ENGINEER],ENGI[28][ENGINEER 

]  ,ENGI[29]  [ENGINEER]  ,ENGI[30]  [ENGINEER], ENGI[3 1]  [ENGINEER]  3NGI[32]  [E 

NGINEER]  ,ENGI[33]  [ENGINEER]  ,ENGI[34]  [ENGINEER]  3NGI[3  5]  [ENGINEER],E 

NGI[36][ENGINEER],ENGI[37][ENGINEER],ENGI[38][ENGINEER],ENGI[39][ENG 

INEER],ENGI[40][ENGINEER],ENGI[41][ENGINEER],ENGI[42][ENGINEER]- 

GL0BA[42][GL0BAL]3NGI[43][ENGINEER] 

Technologies  Attributes: 

TECH[0]  [TECHNO]  ,TECH[  1]  [TECHNO]  ,TECH[2]  [TECHNO]  ,TECH[3][TECHNO]  ,T 
ECH[4][TECHNO],TECH[5][TECHNO],TECH[6][TECHNO],TECH[7][TECHNO],TE 
CH[8][TECHNO],TECH[9][TECHNO],TECH[10][TECHNO],TECH[11][TECHNO],T 
ECH[12][TECHNO],TECH[13][TECHNO],TECH[14][TECHNO],TECH[15][TECHNO 
],TECH[  16]  [TECHNO]  ,TECH[  17]  [TECHNO],TECH[  1 8]  [TECHNO],TECH[19][TECH 
NO]  ,TECH[20]  [TECHNO]  ,TECH[2  l][TECHNO],TECH[22]  [TECHNO]  ,TECH[23]  [TE 
CHNO],TECH[24][TECHNO],TECH[25][TECHNO],TECH[26]  [TECHNO], TECH[27][ 
TECHNO],TECH[28][TECHNO],TECH[29][TECHNO],TECH[30][TECHNO],TECH[ 

3 1]  [TECHNO]  ,TECH[32]  [TECHNO]  ,TECH[33]  [TECHNO]  ,TECH[34]  [TECHNO]  ,TE 
CH[35][TECHNO],TECH[36][TECHNO],TECH[37][TECHNO],TECH[38][TECHNO], 
TECH[39][TECHNO],TECH[40][TECHNO],TECH[41][TECHNO],TECH[42][TECHN 
O]  -ENGI[42]  [ENGINEER]  ,TECH[43]  [TECHNO] 
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C4  Attributes: 

CFOUR[0]  [C4],CF0UR[  1]  [C4],CFOUR[2][C4],CFOUR[3]  [C4],CFOUR[4]  [C4],CF0U 
R[5]  [C4]  ,CF0UR[6]  [C4]  ,CF0UR[7]  [C4]  ,CF0UR[8]  [C4]  ,CF0UR[9]  [C4]  ,CFOUR[  10]  [ 
C4]  ,CFOUR[  1 1]  [C4],CFOUR[  12]  [C4],CFOUR[  13]  [C4]  ,CFOUR[  14]  [C4]  ,CFOUR[  15]  [ 
C4]  ,CFOUR[  1 6]  [C4]  ,CFOUR[  17]  [C4],CFOUR[  1 8]  [C4]  ,CFOUR[  19]  [C4]  ,CFOUR[20]  [ 
C4],CFOUR[21][C4],CFOUR[22][C4],CFOUR[23][C4],CFOUR[24][C4],CFOUR[25][ 
C4]  ,CFOUR[26]  [C4]  ,CFOUR[27]  [C4],CFOUR[28]  [C4]  ,CFOUR[29]  [C4]  ,CFOUR[30]  [ 
C4]  ,CFOUR[3 1]  [C4],CFOUR[32]  [C4],CFOUR[33]  [C4],CFOUR[34][C4],CFOUR[35][ 
C4],CFOUR[36][C4],CFOUR[37][C4],CFOUR[38][C4],CFOUR[39][C4],CFOUR[40][ 
C4]  ,CFOUR[4 1]  [C4]  ,CFOUR[42]  [C4]-TECH[42]  [TECHNO]  ,CFOUR[43]  [C4] 
Management  and  Staff  Attributes: 

MGT[0]  [MGT_STAFF],MGT[  1]  [MGT_STAFF],MGT[2]  [MGT_STAFF],MGT[3]  [MG 

T_STAFF],MGT[4][MGT_STAFF],MGT[5][MGT_STAFF],MGT[6][MGT_STAFF],M 

GT[7]  [MGT_STAFF]  ,MGT[8]  [MGT_STAFF],MGT[9]  [MGT_STAFF],MGT[  10][MGT 

_STAFF],MGT[11][MGT_STAFF],MGT[12][MGT_STAFF],MGT[13][MGT_STAFF], 

MGT[14][MGT_STAFF],MGT[15][MGT_STAFF],MGT[16][MGT_STAFF],MGT[17][ 

MGT_STAFF]  ,MGT[  1 8]  [MGT_STAFF],MGT[19][MGT_STAFF],MGT[20]  [MGT_ST 

AFF],MGT[21][MGT_STAFF],MGT[22][MGT_STAFF],MGT[23][MGT_STAFF],MG 

T[24][MGT_STAFF],MGT[25][MGT_STAFF]JVIGT[26][MGT_STAFF],MGT[27][MG 

T_STAFF],MGT[28][MGT_STAFF],MGT[29][MGT_STAFF],MGT[30][MGT_STAFF] 

,MGT[3 1]  [MGT_STAFF],MGT[32]  [MGT_STAFF]  ,MGT[33]  [MGT_STAFF],MGT[34] 

[MGT_STAFF],MGT[35][MGT_STAFF],MGT[36][MGT_STAFF],MGT[37][MGT_ST 

AFF],MGT[38][MGT_STAFF],MGT[39][MGT_STAFF],MGT[40][MGT_STAFF],MG 

T[4  1]  [MGT_STAFF],MGT[42][MGT_STAFF]- 

CFOUR[42][C4],MGT[43][MGT_STAFF] 

IMINT  Attributes: 

IMIN[0]  [IMINT],IMIN[  1  ]  [IMINT]  ,IMIN[2]  [IMINT]  ,IMIN[3]  [IMINT]  ,IMIN[4]  [IMINT 
],IMIN[5][IMINT],IMIN[6]  [IMINT], IMIN[7]  [IMINT], IMIN[8][IMINT],IMIN[9][IMIN 
T]JMIN[10][IMINT],IMIN[11][IMINT],IMIN[12][IMINT],IMIN[13][IMINT],IMIN[1 
4][IMINT],IMIN[15][IMINT],IMIN[16][IMINT],IMIN[17][IMINT],IMIN[18][IMINT], 
IMIN[19][IMINT],IMIN[20][IMINT],IMIN[21][IMINT],IMIN[22][IMINT],IMIN[23][I 
MINT],IMIN[24][IMINT],IMIN[25][IMINT],IMIN[26][IMINT],IMIN[27][IMINT],IMI 
N[28][IMINT]  ,IMIN[29]  [IMINT]  ,IMIN[30]  [IMINT]  ,IMIN[3 1]  [IMINT]  ,IMIN[32]  [IMI 
NT]  ,IMIN[33][IMINT],IMIN[34][IMINT],IMIN[35][IMINT],IMIN[36]  [IMINT]  ,IMIN[ 
37][IMINT],IMIN[38][IMINT],IMIN[39][IMINT],IMIN[40][IMINT],IMIN[41][IMINT 
]  ,IMIN[42]  [IMINT]  -MGT[42]  [MGT_STAFF],IMIN[43]  [MINT] 
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Space  Systems  Attributes: 

SPACE[0][SPACE_SYS],SPACE[1][SPACE_SYS],SPACE[2][SPACE_SYS],SPACE[3 

]  [SPACE_S  YS]  ,SPACE[4]  [SPACE_S  YS]  ,SPACE[5]  [SPACE_S  YS]  ,SPACE[6]  [SPACE 

_SYS],SPACE[7][SPACE_SYS],SPACE[8][SPACE_SYS],SPACE[9][SPACE_SYS],S 

PACE[10][SPACE_SYS],SPACE[111[SPACE_SYS],SPACE[12][SPACE_SYS],SPAC 

E[13][SPACE_SYS],SPACE[14][SPACE_SYS],SPACE[15][SPACE_SYS],SPACE[16] 

[SPACE_SYS],SPACE[17][SPACE_SYS],SPACE[18][SPACE_SYS],SPACE[19][SPA 

CE_SYS],SPACE[20][SPACE_SYS],SPACE[21][SPACE_SYS],SPACE[22][SPACE_S 

YS],SPACE[23][SPACE_SYS],SPACE[24][SPACE_SYS],SPACE[25][SPACE_SYS],S 

PACE[26][SPACE_SYS],SPACE[27][SPACE_SYS],SPACE[28][SPACE_SYS],SPAC 

E[29]  [SPACE_S  YS]  ,SPACE[30]  [SPACE_SYS],SPACE[3 1]  [SPACE_SYS],SPACE[32] 

[SPACE_SYS],SPACE[33][SPACE_SYS],SPACE[34][SPACE_SYS],SPACE[35][SPA 

CE_SYS],SPACE[36][SPACE_SYS],SPACE[37][SPACE_SYS],SPACE[38][SPACE_S 

YS],SPACE[39][SPACE_SYS],SPACE[40][SPACE_SYS],SPACE[41][SPACE_SYS],S 

PACE[42][SPACE_SYS]-IMIN[42][IMINT],SPACE[43][SPACE_SYS] 

Publications  Attributes: 

PUB[0][PUBS],PUB[1][PUBS],PUB[2][PUBS],PUB[3][PUBS],PUB[4][PUBS],PUB[5] 
[PUBS],PUB[6][PUBS],PUB[7][PUBS],PUB[8][PUBS],PUB[9][PUBS],PUB[10][PUB 
S],PUB[11][PUBS],PUB[12][PUBS],PUB[13][PUBS]^UB[14][PUBS],PUB[15][PUBS 
],PUB[16][PUBS],PUB[17][PUBS],PUB[18][PUBS],PUB[19][PUBS],PUB[20][PUBS], 
PUB[21][PUBS],PUB[22][PUBS],PUB[23][PUBS],PUB[24][PUBS],PUB[25][PUBS],P 
UB[26]  [PUBS]  ,PUB  [27]  [PUBS]  ,PUB[28]  [PUBS]^UB  [29]  [PUBS], PUB  [30]  [PUBS],P 
UB[3 1]  [PUBS], PUB  [32]  [PUBS], PUB  [33]  [PUBS], PUB  [34]  [PUBS],PUB  [35]  [PUBS],P 
UB[36]  [PUBS]  ,PUB  [37]  [PUBS]  ,PUB  [38]  [PUBS]  ,PUB  [39]  [PUBS], PUB  [40]  [PUBS],P 
UB[41][PUBS],PUB[42][PUBS]-SPACE[42][SPACE_SYS],PUB[43][PUBS] 
Requirements  Attributes: 

REQ[0][REQUIRE],REQ[1][REQUIRE],REQ[2][REQUIRE],REQ[3][REQUIRE],REQ 
[4]  [REQUIRE]  ,REQ[5]  [REQUIRE], REQ[6]  [REQUIRE], REQ[7]  [REQUIRE]  ,REQ[8]  [ 
REQUIRE],REQ[9][REQUIRE],REQ[10][REQUIRE],REQ[11][REQUIRE],REQ[12][R 
EQUIRE],REQ[  13][REQUIRE]  ,REQ[  14]  [REQUIRE]  ,REQ[  15]  [REQUIRE]  ,REQ[  16]  [R 
EQUIRE],REQ[  17]  [REQUIRE]  ,REQ[1 8]  [REQUIRE]  ,REQ[19]  [REQUIRE]  ,REQ[20]  [R 
EQUIRE],REQ[21]  [REQUIRE]  ,REQ[22]  [REQUIRE]  ,REQ[23]  [REQUIRE]  ,REQ[24]  [R 
EQUIRE],REQ[25][REQUIRE],REQ[26][REQUIRE],REQ[27][REQUIRE],REQ[28][R 
EQUIRE],REQ[29][REQUIRE]  ,REQ[30]  [REQUIRE]  ,REQ[3 1]  [REQUIRE]  ,REQ[32]  [R 
EQUIRE],REQ[33]  [REQUIRE]  ,REQ[34][REQUIRE]  ,REQ[35]  [REQUIRE]  ,REQ[36]  [R 
EQUIRE],REQ[37]  [REQUIRE]  ,REQ[38][REQUIRE]  ,REQ[39]  [REQUIRE]  ,REQ[40]  [R 
EQUIRE],REQ[41]  [REQUIRE]  ,REQ[42][REQUrRE]- 
PUB[42][PUBS],REQ[43][REQUIRE] 
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Intelligence  Analysis  Attributes: 

INTEL[0]  [INTEL_AN],INTEL[  1]  [INTEL_AN],INTEL[2][INTEL_AN]  ,INTEL[3]  [INT 

EL_AN],INTEL[4][INTEL_AN],INTEL[5][INTEL_AN],INTEL[6][INTEL_AN],INTE 

L[7][INTEL_AN],INTEL[8][INTEL_AN],INTEL[9][INTEL_AN],INTEL[10][INTEL_ 

AN], INTEL[11][INTEL_AN],INTEL[12][INTEL_AN],INTEL[13][INTEL_AN], INTEL 

[14][INTEL_AN],INTEL[15][INTEL_AN],INTEL[16][INTEL_AN],INTEL[17][INTEL 

_AN],INTEL[18][INTEL_AN],INTEL[19][INTEL_AN],INTEL[20][INTEL_AN],INTE 

L[21][INTEL_AN],INTEL[22][INTEL_AN],INTEL[23][INTEL_AN],INTEL[24][INTE 

L_AN],INTEL[25][INTEL_AN],INTEL[26][INTEL_AN],INTEL[27][INTEL_AN],INT 

EL[28]  [INTEL_AN]  ,INTEL[29]  [INTEL_AN],INTEL[30]  [INTEL_AN]  ,INTEL[3 1]  [INT 

EL_AN],INTEL[32][INTEL_AN],INTEL[33][INTEL_AN],INTEL[34][INTEL_AN],IN 

TEL[35][INTEL_AN],INTEL[36][INTEL_AN],INTEL[37][INTEL_AN],INTEL[38][IN 

TEL_AN],INTEL[39][INTEL_AN],INTEL[40][INTEL_AN],INTEL[41][INTEL_AN],I 

NTEL[42][INTEL_AN]-REQ[42][REQUIRE],INTEL[43][INTEL_AN] 

Production  Operations  Attributes: 

PROD[0][PROD_OPS],PROD[1][PROD_OPS],PROD[2][PROD_OPS],PROD[3][PRO 

D_OPS],PROD[4][PROD_OPS],PROD[5][PROD_OPS],PROD[6][PROD_OPS],PROD[ 

7][PROD_OPS],PROD[8][PROD_OPS],PROD[9][PROD_OPS],PROD[10][PROD_OP 

S]T’ROD[ll][PROD_OPS],PROD[12][PROD_OPS],PROD[13][PROD_OPS],PROD[14 

]  [PROD_OPS],PROD[15][PROD_OPS],PROD[16][PROD_OPS],PROD[  17]  [PROD_OP 

S]J>ROD[18][PROD_OPS]T‘ROD[19][PROD_OPS],PROD[20][PROD_OPS].PROD[21 

][PROD_OPS],PROD[22][PROD_OPS],PROD[23][PROD_OPS],PROD[24][PROD_OP 

S]J»ROD[25][PROD_OPS],PROD[26][PROD_OPS],PROD[27][PROD_OPS],PROD[28 

][PROD_OPS],PROD[29][PROD_OPS],PROD[30][PROD_OPS]JPROD[31][PROD_OP 

S],PROD[32][PROD_OPS],PROD[33][PROD_OPS],PROD[34][PROD_OPS],PROD[35 

][PROD_OPS],PROD[36][PROD_OPS],PROD[37][PROD_OPS],PROD[38][PROD_OP 

S]  T>ROD[39]  [PROD_OPS]  ,PROD[40]  [PROD_OPS]  ,PROD[4 1]  [PROD_OPS]  ,PROD[42 

][PROD_OPS]-INTEL[42][INTEL_AN],PROD[43][PROD_OPS] 

Ballistic  Missiles  Attributes: 

BAL[0][BAL_MIS],BAL[1][BAL_MIS],BAL[2][BAL_MIS],BAL[3][BAL_MIS],BAL[ 

4][BAL_MIS],BAL[5][BAL_MIS],BAL[6][BAL_MIS],BAL[7][BAL_MIS],BAL[8][BA 

L_MIS],BAL[9][BAL_MIS],BAL[10][BAL_MIS],BAL[11][BAL_MIS],BAL[12][BAL_ 

MIS],BAL[13][BAL_MIS],BAL[14][BAL_MIS],BAL[15][BAL_MIS],BAL[16][BAL_ 

MIS],BAL[17][BAL_MIS],BAL[18][BAL_MIS],BAL[19][BAL_MIS],BAL[20][BAL_ 

MIS],BAL[21][BAL_MIS],BAL[22][BAL_MIS],BAL[23][BAL_MIS],BAL[24][BAL_ 

MIS],BAL[25][BAL_MIS],BAL[26][BAL_MIS],BAL[27][BAL_MIS],BAL[28][BAL_ 

MIS],BAL[29][BAL_MIS],BAL[30][BAL_MIS],BAL[31][BAL_MIS],BAL[32][BAL_ 

MIS],BAL[33][BAL_MIS],BAL[34][BAL_MIS],BAL[35][BAL_MIS],BAL[36][BAL_ 

MIS],BAL[37][BAL_MIS],BAL[38][BAL_MIS],BAL[39][BAL_MIS],BAL[40][BAL_ 

MIS]  ,B  AL[4 1  ]  [B  AL_MIS]  ,B  AL[42]  [BAL_MIS]- 

PROD[42][PROD_OPS],BAL[43][BAL_MIS] 
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Combat  Intelligence  Attributes: 

COM[0][COM_INT],COM[1][COM_INT],COM[2][COM_INT],COM[3][COM_INT],C 
OM[4]  [COMJNT]  ,COM[5][COM_INT],COM[6]  [COMJNT]  ,COM[7]  [COM_INT|  ,CO 
M[8][COM_INT],COM[9][COM_INT],COM[10][COM_INT],COM[11][COMJNT],C 
OM[12][COM_INT],COM[13][COM_INT],COM[14][COM_INT],COM[15][COM_IN 
T],COM[  1 6]  [COM_INT],COM[  17][COM_INT],COM[  1 8][COM_INT],COM[  19]  [COM 
_INT],COM[20]  [COMJNT]  ,COM[21][COM_INT],COM[22]  [COM_INT],COM[23]  [C 
OM_INT],COM[24][COM_INT],COM[25][COM_INT],COM[26][COM_INT],COM[27 
]  [COM_INT]  ,COM[28]  [COM_INT]  ,COM[29][COM_INT]  ,COM[30]  [COM JNT],COM 
[31][COM_INT],COM[32][COM_INT],COM[33][COM_INT],COM[34][COM_INT],C 
OM[35][COM_INT],COM[36][COM_INTi,COM[37][COM_INT],COM[38][COM_IN 
T],COM[39][COM_INT],COM[40][COM_INT],COM[41][COM_INT],COM[42][COM 
_INT]-BAL[42]  [BAL_MIS],COM[43]  [COMJNT] 
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Appendix  G.  Constraints  and  Objective  Function 


Constraints: 

(($43+$44)  <=  22927.5  ?  0  :  halt(-2))  &&  ($42  <=  2547.5  ?  0  :  halt(-l)) 


Objective  Function: 

(($43+$44)<=22927.5  &&  ($42  <=  2547.5)  ? 
(D1*(C1*(B4*A1*$1/130+B5*A4*$4/130+B6*(A7*$7/130+A8*$8/130+A9*$9/130+A 
10*$10/130))+C2*(B1*A2*$2/130+B2*A5*$5/130+B3*(A11*$11/130+A12*$12/130+ 
A13*$13/130+A14*$14/130))+C3*(B7*A3*$3/130+B8*A6*$6/130+B9*(A15*$15/130 
+A16*$16/130+A17*$17/130+A18*$18/130)))+D2*(A19*$19/130+C6*A20*$20/130+ 
C7*(B11*(A21*$21/130+A22*$22/130)+B12*(A23*$23/130+A24*$24/130+A25*$25/1 
30))+C8*A26*$26/130)+D3*(C9*(A27*$27/130+B13*(A28*$28/130+A29*$29/130)+A 
30*$30/1304B14*(A31*$31/130+A32*$32/130+A33*$33/130)+A34*$34/130)+C10*(B 
15*(A35*$35/130+A36*$36/130)+A37*$37/130+B16*(A38*$38/130+A39*$39/130)+A 
40*$40/130+A41*$4 1/130))) :  0) 
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Appendix  H.  Spreadsheet  Values 


MEASURES 


Throughput(ACQ) 

Accessibility(ACQ) 

Accuracy(ACQ) 

Relevance(ACQ) 

Timeliness(ACQ) 

Throughput(OPS) 

Accessibility(OPS) 

Accuracy  (OPS) 

Relevance(OPS) 

Timeliness(OPS) 

Throughput(PMK) 

Accessibility(PMK) 

Accuracy(PMK) 

Relevance(PMK) 

Timeliness(PMK) 

Futures  Analysis 
Threat  Description 
Counter  Denial/Deception 
Foreign  IW  Capabilities 
C4 

Electronic  Systems 
Space  Systems 
Tech  Assimilation 
Flexibility 
Training 

Community  Work 
#DaysTDY 
OverAJnder  Worked 


DUl-1 


Decision  Unit  Funding  Cut  Codes 


DUl-2  DUl-3  DUl-4 
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Appendix  H.  Spreadsheet  Values 


MEASURES 

Decision  Unit  Funding 

1  Cut  Codes 

DUl-6 

DU2-1 

DU2-2 

DU2-3 

DU2-4 

Throughput(ACQ) 

6.8 

9.7 

9.1 

9.7 

9.3 

Accessibility(ACQ) 

5 

0.7 

0.8 

0.8 

1 

Acciiracy(ACQ) 

6.6 

9.7 

9.8 

9.8 

9.5 

Relevance(ACQ) 

6.3 

9.8 

9.8 

9.8 

9.7 

Timeliness(ACQ) 

10 

9.8 

9.7 

9.8 

9.7 

Throughput(OPS) 

5.8 

9.8 

9.7 

9.8 

9.8 

Accessibility(OPS) 

1.9 

0.9 

0.6 

0.8 

0.9 

Accuracy  (OPS) 

7.7 

9.7 

9.7 

9.8 

9.7 

Relevance(OPS) 

5 

9.8 

9.8 

9.8 

9.8 

Timeliness(OPS) 

10 

9.8 

9.8 

9.8 

9.8 

Throughput(PMK) 

2.8 

9.8 

6.6 

6.2 

9.8 

Accessibility(PMK) 

0 

0.8 

0.6 

0.7 

0.5 

Accuracy  (PMK) 

5 

9.7 

7.9 

6.9 

9.7 

Relevance(PMK) 

5 

9.7 

7.2 

6.9 

9.6 

Timeliness(PMK) 

10 

9.7 

9.8 

9.6 

9.6 

Futures  Analysis 

3.4 

9.8 

9.8 

9.8 

9.8 

Threat  Description 

3.4 

9.8 

9.8 

9.8 

9.9 

Counter  Denial/Deception 

0 

9.7 

9.8 

9.8 

9.8 

Foreign  IW  Capabilities 

0.2 

9.6 

9.8 

9.8 

9.8 

C4 

0 

9.7 

9.8 

9.8 

9.6 

Electronic  Systems 

0 

6.7 

5.8 

6.7 

3.7 

Space  Systems 

0 

9.8 

9.8 

9.6 

8.7 

Tech  Assimilation 

0.1 

9.7 

9.6 

9.6 

9 

Flexibihty 

2.9 

8.1 

8.5 

8.6 

9.4 

Training 

0 

0.8 

0.7 

1.2 

0.7 

Community  Work 

0 

1.8 

1.3 

1.7 

0.7 

#Days  TDY 

0 

2.9 

3.1 

0.8 

Over/Under  Worked 

0 

2.6 

6.7 
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Appendix  H.  Spreadsheet  Values 


MEASURES 

Decision  Unit  Funding  Cut  Codes 

DU3-1 

DU3-2 

DU3-3 

DU3-4 

DU3-5 

Throughput(ACQ) 

3.5 

9.4 

6.2 

5.4 

7.6 

Accessibility(ACQ) 

8.8 

9.3 

6.5 

3.9 

9.4 

Accuracy(ACQ) 

5.6 

7 

5.9 

3.6 

9 

Relevance(ACQ) 

7 

7.9 

6.7 

3.4 

8.7 

Timeliness(ACQ) 

4.6 

8.5 

6 

4.4 

8.1 

Throughput(OPS) 

5.4 

9.4 

6.7 

2.7 

8.4 

Accessibility(OPS) 

9.1 

9.3 

5.7 

3.7 

9.1 

Accuracy(OPS) 

5.8 

8 

5.9 

3.1 

6.3 

Relevance(OPS) 

6.6 

9.5 

6.4 

6.2 

Timeliness(OPS) 

9.8 

7 

8 

8.5 

Throughput(PMK) 

5.8 

4.4 

3.1 

6.2 

Accessibility  (PMK) 

9.6 

9.4 

5.4 

6.1 

5.7 

Accuracy(PMK) 

4.1 

5.6 

4.5 

6.4 

Relevance(PMK) 

5.7 

6 

4 

6.4 

Timeliness(PMK) 

5.2 

8.4 

2.4 

5.4 

6.9 

Futures  Analysis 

6.3 

2.1 

2.2 

6.5 

7.6 

Threat  Description 

8.1 

8.9 

3.6 

6.1 

9.3 

Counter  Denial/Deception 

7.9 

8.9 

3 

5.9 

8.7 

Foreign  IW  Capabilities 

7.9 

9.4 

4.6 

6.3 

8.5 

C4 

6.2 

7.7 

2.6 

5.5 

8.9 

Electronic  Systems 

7.3 

7.5 

7.6 

5.1 

9.3 

Space  Systems 

4.9 

6.5 

1.3 

3.9 

5.2 

Tech  Assimilation 

6.3 

7.9 

5.1 

5.2 

6.6 

Flexibility 

6.9 

3.7 

3 

6.7 

6 

Training 

1.6 

2.7 

2 

6.9 

6.5 

Community  Work 

8.5 

7.2 

3.5 

7 

5.1 

#Days  TDY 

2.6 

4 

1.7 

5.3 

3.4 

OverAJnder  Worked 

3.7 

4.3 

2.7 

4.6 

3.8 
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Appendix  H.  Spreadsheet  Values 


MEASURES 

Decision  Unit  Funding  Cut  Codes 

DU3-6 

DU3-7 

DU4-1 

DU4-2 

DU4-3 

Throughput(ACQ) 

3.8 

8.7 

7.8 

7.9 

7.6 

Accessibility(ACQ) 

2.6 

9.2 

7.3 

7.1 

6.2 

Accuracy(ACQ) 

1.7 

9.4 

10 

7.1 

7.5 

Relevance(ACQ) 

1.8 

9.1 

10 

7.3 

6.8 

Timeliness(ACQ) 

3.5 

9.5 

10 

10 

10 

Throughput(OPS) 

2.7 

9.8 

10 

7.5 

7.4 

Accessibility(OPS) 

1.8 

9.6 

7.2 

7.2 

7.5 

Accuracy  (OPS) 

1.4 

8 

9.9 

7.5 

9 

Relevance(OPS) 

1.2 

9.6 

9.9 

6.5 

6.4 

Timeliness(OPS) 

0.8 

9.2 

10 

10 

10 

Throughput(PMK) 

2.4 

7.3 

7 

8.5 

8.8 

Accessibility(PMK) 

1.9 

7.7 

7.3 

7.1 

6.9 

Accuracy(PMK) 

1.2 

6.5 

10 

6.1 

Relevance(PMK) 

1.3 

6.6 

7.3 

7.9 

Timeliness(PMK) 

2.3 

6.6 

10 

9.8 

Futures  Analysis 

1.8 

9.5 

4.8 

2.8 

Threat  Description 

0.3 

9.5 

4.4 

2.5 

Counter  Denial/Deception 

0.2 

8.4 

6.4 

3 

Foreign  IW  Capabilities 

0.3 

9.4 

6.4 

3.8 

C4 

1.9 

9.7 

1.9 

2.1 

Electronic  Systems 

0.3 

9.7 

9.9 

10 

10 

Space  Systems 

0.4 

4.8 

1.5 

1.9 

1.7 

Tech  Assimilation 

0.3 

8.3 

3.9 

3.5 

3.1 

Flexibility 

2.6 

9.7 

7.9 

6.7 

Training 

3.6 

6.5 

8.3 

9.8 

Community  Work 

3.3 

7.1 

0 

0 

#Days  TDY 

2.5 

4 

1.9 

2.7 

OverAJnder  Worked 

2.8 

8.5 

10 

10 

10 
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Appendix  H.  Spreadsheet  Values 


I 


MEASURES 

Decision  Unit  Funding  Cut  Codes 

DU4-4 

DU4-5 

DU4-6 

DU4-7 

DU5-1 

Throughput(ACQ) 

6.6 

6.4 

4.75 

8.8 

6.3 

Accessibility(ACQ) 

8.9 

6.3 

4.6 

10 

5 

Accuracy(ACQ) 

9.9 

5 

9 

10 

7.5 

Relevance(ACQ) 

7.9 

5 

10 

10 

5 

Timeliness(ACQ) 

7.9 

6.5 

6.6 

10 

5 

Throughput(OPS) 

8.8 

6.6 

10 

10 

0 

Accessibility(OPS) 

10 

5 

7.3 

9.9 

5 

Accuracy(OPS) 

8.3 

5 

10 

10 

8.3 

Relevance(OPS) 

8.3 

6.6 

10 

10 

5 

Timeliness(OPS) 

9.9 

7.9 

10 

10 

5 

ThToughput(PMK) 

7.3 

5.6 

5.1 

7.7 

6 

Accessibility  (PMK) 

9.8 

4.9 

5.1 

10 

5.1 

Accuracy(PMK) 

8.8 

5 

10 

7.9 

7.7 

Relevance(PMK) 

8.3 

7.6 

8.5 

10 

5 

Timeliness(PMK) 

9.9 

8.4 

10 

10 

6.6 

Futures  Analysis 

1.9 

6.9 

2.7 

6.3 

Threat  Description 

2.1 

2.5 

4.8 

6.3 

Counter  Denial/Deception 

3.5 

3.8 

7.1 

3.1 

6.5 

Foreign  IW  Capabilities 

3.5 

2.1 

4.8 

3.1 

1.9 

C4 

3.3 

2.3 

0 

2.1 

1.7 

Electronic  Systems 

10 

9.9 

10 

9.9 

1.5 

Space  Systems 

1.9 

1.8 

1.2 

1.9 

5 

Tech  Assimilation 

3.3 

5.5 

4.6 

3.8 

6.9 

Flexibility 

9.2 

6.9 

4.6 

6.7 

7 

Training 

10 

7.1 

8.3 

10 

1.5 

Community  Work 

0 

0 

0 

0 

0.3 

#Days  TDY 

4.8 

2.2 

1.5 

4 

0.3 

OverAJnder  Worked 

10 

7.1 

10 

8.9 

4.8 
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Appendix  H.  Spreadsheet  Values 


MEASURES 

Decision  Unit  Funding  Cut  Codes 

DU5-2 

DU5-3 

DU6-1 

DU6-2 

DU6-3 

Throughput(ACQ) 

5.2 

5.1 

3 

4 

0.4 

Accessibility(ACQ) 

5 

5.1 

0 

5 

0 

Accuracy(ACQ) 

7.7 

6.9 

5 

7.6 

7.5 

Relevance(ACQ) 

5 

5.1 

0 

8.7 

0.6 

Timeliness(ACQ) 

5.2 

10 

10 

10 

3.5 

Throughput(OPS ) 

5.2 

5 

0.2 

3.5 

2.7 

Accessibility(OPS) 

5 

5 

0 

5.1 

0 

Accuracy(OPS) 

6.9 

7.7 

5 

6.5 

8.5 

Relevance(OPS) 

5.2 

5 

0.2 

4.6 

8.6 

Timeliness(OPS) 

10 

10 

0.5 

10 

10 

Throughput(PMK) 

5.2 

5.1 

3 

3.7 

0.8 

Accessibility  (PMK) 

5 

5.2 

0 

5 

0 

Accuracy(PMK) 

6.7 

7.7 

5 

6.9 

7.5 

Relevance(PMK) 

5 

5.1 

2.8 

8.3 

0.6 

Timeliness(PMK) 

10 

10 

5.1 

10 

4 

Futures  Analysis 

5.2 

7.4 

1.5 

4.8 

0 

Threat  Description 

5.2 

5 

0 

5 

5 

Counter  Denial/Deception 

5.4 

5.1 

0 

1.6 

0 

Foreign  IW  Capabilities 

2.9 

5.2 

0 

1.5 

0 

C4 

0.8 

0.5 

0 

1.5 

4.8 

Electronic  Systems 

3.3 

2.6 

0 

6.6 

4.6 

Space  Systems 

4.8 

0.3 

0.7 

0 

Tech  Assimilation 

7.1 

2 

0.4 

5 

Flexibility 

6 

7.5 

2.7 

4.8 

Training 

1 

0.5 

0 

2.1 

0 

Community  Work 

0.4 

0.4 

0 

0 

0 

#Days  TDY 

0.4 

0.4 

5 

2.4 

4.8 

OverAJnder  Worked 

3.8 

3 

9.8 

10 
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Appendix  H.  Spreadsheet  Values 


MEASURES 


Throughput(ACQ) 

Accessibility(ACQ) 

Accuracy(ACQ) 

Relevance(ACQ) 

Timeliness(ACQ) 

Throughput(OPS) 

Accessibility(OPS) 

Accuracy(OPS) 

Relevance(OPS) 

Timeliness(OPS) 

Throughput(PMK) 

AccessibilityCPMK) 

Accuracy(PMK) 

Relevance  (PMK) 

Tinieliness(PMK) 

Futures  Analysis 
Threat  Description 
Counter  Denial/Deception 
Foreign  VN  Capabilities 
C4 

Electronic  Systems 
Space  Systems 
Tech  Assimilation 
Flexibility 
Training 

Community  Work 
#Days  TDY 
Over/Under  Worked 


Appendix  H.  Spreadsheet  Values 


I 


MEASURES 

Decision  Unit  Funding  Cut  Codes 

DU8-1 

DU8-2 

DU8-3 

DU8-4 

DU8-5 

Throughput(ACQ) 

6.9 

10 

8.3 

7.1 

9.6 

Accessibility(ACQ) 

5 

5 

5 

10 

5 

Accuracy(ACQ) 

6.7 

10 

8.5 

8.7 

8.1 

Relevance(ACQ) 

6.9 

10 

5.1 

5.1 

5 

Timeliness(ACQ) 

10 

10 

10 

10 

10 

Throughput(OPS) 

4.4 

10 

7.7 

8.5 

9.2 

Accessibility(OPS) 

5 

5.1 

5 

10 

5.1 

Accuracy  (OPS) 

7.3 

10 

8.1 

7.7 

Relevance(OPS) 

5 

10 

5.2 

5 

Timeliiiess(OPS) 

10 

10 

10 

10 

10 

Throughput(PMK) 

7.3 

10 

8.1 

8.4 

9 

Accessibility(PMK) 

5 

5 

5 

10 

5 

Accuracy(PMK) 

7.9 

10 

8.3 

6.7 

Relevance(PMK) 

5.6 

10 

5 

5.2 

Timeliness(PMK) 

9.8 

9.9 

10 

10 

Futures  Analysis 

9 

10 

9 

9.4 

Threat  Description 

4.8 

2.8 

5.6 

7.5 

Counter  Denial/Deception 

5 

5.2 

7.3 

Foreign  IW  Capabilities 

9.4 

5.2 

7.6 

7.3 

C4 

4.4 

5.1 

6.9 

6.3 

Electronic  Systems 

5 

4.8 

7.2 

6.5 

Space  Systems 

4.8 

5.8 

7.3 

6.5 

Tech  Assimilation 

7.9 

8.5 

8.3 

6.8 

8.5 

Flexibility 

5.8 

10 

10 

7.1 

10 

Training 

0 

0 

0 

0 

0.1 

Community  Work 

0 

0 

0 

0 

1 

#DaysTDY 

0 

0 

0 

0 

0 

OverAJnder  Worked 

8.6 

0 

10 

9.9 

10 

105 


Appendix  H.  Spreadsheet  Values 


MEASURES 

Decision  Unit  Funding  Cut  Codes 

DU9-1 

DU9-2 

DU9-3 

DU9-4 

DU9-5 

Throughput(ACQ) 

2.1 

1.3 

4.8 

Accessibility(ACQ) 

1.9 

n 

1.1 

5 

Accuracy(ACQ) 

5 

7.5 

2.2 

6.9 

Relevance(ACQ) 

0 

2.1 

5 

0 

5 

Timeliness(ACQ) 

0 

2.2 

4.6 

0 

7.1 

Throughput(OPS ) 

1.4 

2.4 

2.7 

1 

8.2 

Accessibility(OPS) 

1.5 

0 

5 

0 

5.1 

Accuracy(OPS) 

3.5 

2.3 

5 

1 

6.8 

Relevance(OPS) 

0 

1.8 

5 

0 

5 

Timeliness(OPS) 

0 

0 

10 

0 

6.3 

Thioughput(PMK) 

3.3 

4.5 

5.1 

4.7 

7.7 

Acces  sibility  (PMK) 

0 

2 

7.3 

6.5 

7.4 

Accuracy(PMK) 

3.2 

2.6 

7.4 

7.9 

7.2 

Relevance(PMK) 

0 

2.4 

6.5 

5 

5 

Timeliness(PMK) 

2.4 

2.2 

6.5 

2.2 

2.9 

Futures  Analysis 

0 

0 

0 

0 

Threat  Description 

0 

1.9 

2.4 

2.5 

4.6 

Counter  Denial/Deception 

0 

0 

2.6 

2.3 

4.8 

Foreign  IW  Capabilities 

0 

0 

0 

0 

C4 

0 

0 

0 

0 

Electronic  Systems 

0 

0 

0 

0.1 

Space  Systems 

1.3 

1.9 

2.8 

4.9 

4.8 

Tech  Assimilation 

1.4 

1.6 

4.4 

4.8 

4.8 

Flexibility 

0 

0 

2.2 

0 

0 

Training 

0 

0 

0 

0 

Community  Work 

0 

0 

0.1 

0 

#Days  TDY 

0 

0 

0.1 

4.8 

Over/Under  Worked 

1.9 

0 

3.4 

3.5 

4.5 
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Appendix  H.  Spreadsheet  Values 


MEASURES 

Decision  Unit  Funding  Cut  Codes 

DU9-6 

DU9-7 

DU9-8 

DU9-9 

DU9-10 

Throughput(ACQ) 

8.3 

2.5 

2.3 

■9 

1.7 

Accessibility(ACQ) 

10 

5 

2.4 

mm 

5 

Accuracy(ACQ) 

10 

5 

5 

0 

2.2 

Relevance(ACQ) 

7.9 

7.3 

5 

1.7 

2.1 

Timeliness(ACQ) 

10 

7.3 

6.4 

1.1 

3 

Throughput(OPS ) 

10 

6.9 

4.6 

0 

3.9 

Accessibility(OPS) 

0.1 

5 

2.3 

0 

2.3 

Accuracy(OPS) 

2.5 

7.3 

0 

5 

Relevance(OPS) 

2.3 

5.1 

0 

1.9 

Timeliness(OPS) 

0.1 

0 

5.1 

0.1 

4.2 

Throughput(PMK) 

2.5 

2.5 

7 

4.6 

6.2 

Accessibility  (PMK) 

5 

7.1 

5 

5.1 

5 

Accuracy(PMK) 

5 

5 

7.2 

2.7 

6.7 

Relevance(PMK) 

5 

5 

5 

5 

5 

Timeliness(PMK) 

2.4 

2.3 

6.7 

6.3 

6.5 

Futures  Analysis 

0 

0.1 

0 

0 

0 

Threat  Description 

4.7 

2.3 

2.2 

0 

2.2 

Counter  Denial/Deception 

1.9 

2.7 

4.8 

2.5 

4.7 

Foreign  IW  Capabilities 

0 

0 

0 

0 

1.2 

C4 

0 

0.1 

0 

0 

0 

Electronic  Systems 

0 

0 

0 

0 

0 

Space  Systems 

0 

2.1 

0 

0 

Tech  Assimilation 

2.7 

4.2 

2.3 

4.8 

Rexibility 

1.8 

4.6 

2.5 

1.7 

Training 

0 

0 

0.1 

0.1 

Community  Work 

0 

0.1 

0.1 

0 

#Days  TDY 

2.1 

2.1 

4.2 

1.8 

Over/Under  Worked 

4.4 

4.8 

4.4 

2.8 
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Appendix  H.  Spreadsheet  Values 


Decision  Unit  Funding  Cut  Codes 


MEASURES 

DU9-11 

DU9-12 

DU9-13 

DU9-14 

DU9-15 

Throughput(ACQ) 

4.8 

4.7 

0 

0 

0 

Accessibility(ACQ) 

5 

5 

0 

0 

0 

Accuracy(ACQ) 

6 

7.4 

0 

0 

0 

Relevance(ACQ) 

6 

7.3 

0 

0 

0 

Timeliness  (ACQ) 

4.8 

7 

0 

0 

0 

Throughput(OPS) 

6 

7.1 

1.5 

4.8 

5 

Accessibility(OPS) 

5 

9 

1.5 

5 

5 

Accuracy(OPS) 

6.3 

9 

1.5 

5 

5 

Relevance(OPS) 

6 

7.5 

1 

5 

5 

Timeliness(OPS) 

6.3 

7.5 

1 

4.8 

4.8 

Throughput(PMK) 

8.3 

5.1 

1 

7.1 

5 

Accessibility(PMK) 

8.3 

5 

1 

6.9 

5 

Accuracy(PMK) 

8.5 

6.7 

1 

7.2 

5 

Relevance(PMK) 

8.3 

6.6 

1.3 

7.1 

5 

Timeliness(PMK) 

5.2 

5 

1.5 

7.4 

2.9 

Futures  Analysis 

0 

1.3 

0 

0 

0 

Threat  Description 

5.2 

7.3 

2.7 

4.6 

2.5 

Counter  Denial/Deception 

5 

7.6 

0 

2.1 

1.1 

Foreign  IW  Capabilities 

0.1 

1 

0 

0 

0 

C4 

0.1 

1.2 

0 

0 

0 

Electronic  Systems 

0 

1.4 

0 

0 

0 

Space  Systems 

5.2 

5.1 

0 

4.6 

0 

Tech  Assimilation 

7.5 

2.1 

2.7 

Hexibility 

4.8 

0 

2.3 

Training 

0 

0 

0 

0 

Community  Work 

0 

0 

0 

0 

#Days  TDY 

5 

0 

4.8 

Over/Under  Worked 

6.9 

0 

2.1 
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Appendix  H.  Spreadsheet  Values 


Decision  Unit  Fundini 

1  Cut  Codes 

MEASURES 

DU9-16 

DU9-17 

DU9-18 

DU9-19 

DU9-20 

Throughput(ACQ) 

0.8 

1.3 

2 

1.5 

7 

Accessibility(ACQ) 

0.8 

1.3 

1.8 

0 

6.9 

Accuracy(ACQ) 

0.8 

1.3 

2 

1.3 

8.6 

Relevance(ACQ) 

0.8 

1.1 

0 

1.5 

8.5 

Timeliness(ACQ) 

0.9 

1.3 

3.5 

1.3 

10 

Throughput(OPS ) 

5 

4.9 

1.8 

4.6 

8.2 

Accessibility(OPS) 

5 

5 

1.8 

4.8 

1.2 

Accuracy(OPS) 

5 

5 

1.8 

5 

1.2 

Relevance(OPS) 

5 

5 

0 

5 

1 

Timeliness(OPS) 

5 

4.8 

3.8 

5 

0.1 

Throughput(PMK) 

7.6 

7.5 

1.5 

1 

0.9 

Accessibility(PMK) 

8 

7.5 

1.7 

0 

0.9 

Accuracy(PMK) 

8 

7.7 

1.6 

0 

0 

Relevance(PMK) 

8 

7.3 

0 

0 

0.8 

Timeliness(PMK) 

8 

7.3 

3.9 

0.8 

0 

Futures  Analysis 

0 

0 

0 

0 

0 

Threat  Description 

2.1 

4.8 

1.8 

4.8 

1.5 

Counter  Denial/Deception 

4.5 

5 

1.8 

1.9 

1.6 

Foreign  IW  Capabilities 

0 

1.7 

0 

0.1 

0 

C4 

0 

1.9 

0 

0 

0 

Electronic  Systems 

0 

2.3 

0 

0 

0 

Space  Systems 

0 

0 

1.4 

0.1 

0 

Tech  Assimilation 

5 

7.5 

3.3 

4.6 

2.6 

Flexibility 

1.3 

1.5 

4.8 

1.5 

4.6 

Training 

0 

0.1 

4.9 

0 

1 

Community  Work 

0.1 

0.1 

0 

0 

nIH 

#Days  TDY 

0 

4.8 

0 

0 

WBM 

OverAJnder  Worked 

2.3 

4.8 

8.1 

1.1 

mum 

Appendix  H.  Spreadsheet  Values 


MEASURES 

Decision  Unit  Funding  Qit  Codes 

DU9-21 

DUlO-1 

DUlO-2 

Throughput(ACQ) 

3.5 

8.4 

9.3 

Accessibility(ACQ) 

5 

6.9 

0 

Accuracy(ACQ) 

5 

8.4 

9.3 

Relevance(ACQ) 

5 

8.5 

7.3 

Timeliness(ACQ) 

5.2 

10 

9.2 

Throughput(OPS) 

3.5 

8.7 

9 

Accessibility(OPS) 

1.1 

8.9 

7.5 

Accuracy  (OPS) 

5 

7.6 

8.9 

Relevance(OPS) 

1 

7.5 

7.7 

Tiineliness(OPS) 

1.3 

9.9 

0 

Throughput(PMK) 

4.8 

8.3 

9.1 

Acces  sibility  (PMK) 

5 

6.6 

7.7 

Accuracy(PMK) 

5 

4.6 

8.9 

Relevance(PMK) 

5 

9 

7.7 

Timeliness(PMK) 

5.1 

9.9 

0 

Futures  Analysis 

1.7 

8 

9.5 

Threat  Description 

4.4 

6.9 

9.4 

Counter  Denial/Deception 

1.7 

0.2 

9.5 

Foreign  IW  Capabilities 

1.7 

0.2 

9.5 

C4 

0 

0.8 

9.4 

Electronic  Systems 

0 

9.9 

9.2 

Space  Systems 

0 

8.1 

9.2 

Tech  Assimilation 

4.6 

4.5 

9.2 

Flexibility 

1.3 

9.8 

8.4 

Training 

0.1 

5.8 

0 

Community  Work 

0 

6.1 

0.2 

#Days  TDY 

3.5 

9.5 

0 

OverAJnder  Worked 

4.4 

9.7 

9.4 

no 


Appendix  I.  Logical  Decision  Value  Hierarchy 


Appendix  I.  Logical  Decisions  Value  Hierarchy 
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Appendix  I.  Logical  Decisions  Value  Hierarchy 
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Appendix  J.  LINDO  Code 


MIN  2.279  XI  +  0.435  X2  +  2.041  X3  +  4.469  X4  +  0.837  X5 
+  1.839  X6  +  1.971  X7  +  0.478  X8  +  1.066  X9  +  2.723  XIO 
+  0.884  Xll  +  1.199  X12  +  2.986  X13  +  2.959  X14  +  4.078  X15 
+  1.591  X16  +  3.228  X17  +  1.123  X18  +  2.255  X19  +  3.908  X20 
+  3.97  X21  +  4.438  X22  +  2.61  X23  +  5.395  X24  +  3.285  X25 
+  4.757  X26  +  3.247  X27  +  1.232  X28  +  4.968  X29  +  5.534  X30 
+  3.317  X31  +  3.834  X32  +  4.628  X33  +  3.288  X34  +  2.125  X35 
+  5.024  X36  +  2.741  X37  +  3.633  X38  +  1.509  X39  +  4.576  X40 
+  4.558  X41  +  3.348  X42  +  2.05  X43  +  4.561  X44  +  4.938  X45 
+  2.736  X46  +  1.402  X47  +  3.529  X48  +  3.214  X49  +  5.232  X50 
+  2.614  X51  +  5.507  X52  +  3.2  X53  +  5.234  X54  +  5.377  X55 
+  5.003  X56  +  2.401  X57  +  3.812  X58  +  5.372  X59  +  4.326  X60 
+  5.25  X61  +  3.382X62 

SUBJECT  TO 

2)  1396  XI  +  233  X2  +  932  X3  +  1834  X4  +  326  X5  +  698  X6 
+  716  X7  +  173  X8  +  384  X9  +  931  XIO  +  281  XI 1  +  298  X12 
+  714  X13  +  696  X14  +  933  X15  +  359  X16  +  671  X17  +  218  X18 
+  417  X19  +  699  X20  +  698  X21  +  775  X22  +  445  X23  +  903  X24 
+  524  X25  +  758  X26  +  495  X27  +  187  X28  +  735  X29  +  791  X30 
+  465  X31  +  512  X32  +  567  X33  +  390  X34  +  218  X35  +  463  X36 
+  247  X37  +  322  X38  +  130  X39  +  378  X40  +  372  X41  +  257  X42 
+  154  X43  +  342  X44  +  347  X45  +  182  X46  +  93  X47  +  233  X48 
+  211  X49  +  311  X50  +  130  X51  +  261  X52  +  135  X53  +  218  X54 
+  208  X55  +  192  X56  +  70  X57  +  1 10  X58  +  1 10  X59  +  87  X60 
+  87  X61  +  24  X62  >=  8544 

END 

INTE  62 
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Appendix  J.  LINDO  Code 


VARIABLE 

PROPOSAL 

IMPACT(bi) 

FUNDING(Ci) 

XI 

DU9-17 

2.279 

1396 

X2 

DU9-13 

0.435 

233 

X3 

DU9-16 

2.041 

932 

X4 

DU8-5 

4.469 

1834 

X5 

DU9-9 

0.837 

326 

X6 

DU9-10 

1.839 

698 

XI 

DU9-14 

1.971 

716 

X8 

DU9-1 

0.478 

173 

X9 

DU9-2 

1.066 

384 

XIO 

DU9-8 

2.723 

931 

Xll 

DU9-4 

0.884 

281 

X12 

DU3-6 

1.199 

298 

X13 

DU9-3 

2.986 

714 

X14 

DU9-5 

2.959 

696 

X15 

DU8-1 

4.078 

933 

X16 

DU9-15 

1.591 

359 

X17 

DU3-3 

3.228 

671 

X18 

DU6-4 

1.123 

218 

X19 

DU9-7 

2.255 

417 

X20 

DU3-1 

3.908 

699 

X21 

DU9-12 

3.970 

698 

X22 

DUlO-2 

4.438 

775 

X23 

DUl-2 

2.610 

445 

X24 

DU2-4 

5.395 

903 

X25 

DU5-1 

3.285 

524 

X26 

DU3-5 

4.757 

758 

X27 

DUl-3 

3.247 

495 

X28 

DU9-18 

1.232 

187 

X29 

DU4-6 

4.968 

735 

X30 

DU3-7 

5.534 

791 

X31 

DU9-11 

3.317 

465 

X32 

DU6-2 

3.834 

512 

X33 

DU3-2 

4.628 

567 

X34 

DUl-1 

3.288 

390 

X35 

DU9-20 

2.125 

218 

115 


X36 

DU  10-1 

5.024 

463 

X37 

DU6-3 

2.741 

247 

X38 

DU5-3 

3.633 

322 

X39 

DU6-1 

1.509 

130 

X40 

DU4-3 

4.576 

378 

X41 

DU8-4 

4.558 

372 

X42 

DU6-7 

3.348 

257 

X43 

DU9-21 

2.050 

154 

X44 

DU4-2 

4.561 

342 

X45 

DU7-1 

4.938 

347 

X46 

DU9-6 

2.736 

182 

X47 

DU9-19 

1.402 

93 

X48 

DU5-2 

3.529 

233 

X49 

DUl-6 

3.214 

211 

X50 

DU2-3 

5.232 

311 

X51 

DUl-5 

2.614 

130 

X52 

DU4-7 

5.507 

261 

X53 

DU3-4 

3.200 

135 

X54 

DU2-2 

5.234 

218 

X55 

DU2-1 

5.377 

208 

X56 

DU4-4 

5.003 

192 

X57 

DU6-5 

2.401 

70 

X58 

DU4-5 

3.812 

110 

X59 

DU4-1 

5.372 

110 

X60 

DU8-3 

4.326 

87 

X61 

DU8-2 

5.250 

87 

X62 

DU6-6 

3.382 

24 
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Appendix  K.  Sensitivity  Analysis 


Customer  Support  .5  Future  Requirements  .125  Unit  Peifonnance  .375 

Dynamic  Sensitivity  of  Commanders  Values  Ranking 


Alten&tive 

DU9_13 

DU9_1 

DU9J 

DU9_4 

DU9_2 

DU6_4 

DU3_6 

DU9_18 

DU9_19 

DU6_1 

DU9_15 

DU9_10 

DU9J6 

DU9J4 

DU9J11 

DU9J0 

DU9_7 

DU9_17 

DU6  5 

DU1_5 

DU1_2 

DU9_fi 

DU9_8 

DU6J 

nU9_3 

DU9J 

DUIJ 

DUU 

DUU 

DUfi_7 

DU3.3 


Uimty 

0.399125 

0.418075 

0.780888 

w 

0.818363 

0.918412 

0,997350 

1.150950 

1241388 

*.wv.:.: 

1265275 

1381700 

1.418475 

1.632963 

1.781525 

1.787650 

g;'  ' 

1.855175 

1.893050 

g.xxs 

2.019238 

1076013 

1163550 

1202950 

2204788 

StX’  X" 

2397338 

g-iSSW 

1422975 

1523563 

1588000 

ggi  g-sS 

1594488 

I  i 

1782887 

1  m 

1809138 

sgww 

1811613 

1908325 

s  'i 

1924325 

■MM 

Altenulive 

DU6_6 

DU9_n 

DU5.1 

DU3_4 

DU5_2 

DU5_3 

DU6_2 

DU4_5 

DU9_12 

DU3_! 

DU8_1 

DU8.3 

DU8_5 

DU8_4 

DU4_2 

DU10_2 

0U4_3 

DU3Jt 

DU3_S 

DU4_6 

DU4_4 

DU10_1 

DU8_2 

DU7_1 

DU2^ 

DU2_3 

DU4_1 

DU3L1 

DU4^7 

WXA 

DU3,7 


2.946150 

2.981200 

3.074463 

3.147625 

3.231175 

3.287363 

3A60387 

3385338 

3.631813 

3.694488 

3.791438 

4.043063 

4224325 

4225238 

4228013 

4241263 

4242800 

4273750 

4314913 

4337275 

4.619200 

4.698075 

4.749325 

4.757038 

4.907600 

4.911588 

4.944675 

5.011225 

5.032825 

5.046288 

5274963 


f  ' 


^  'I 

'  X  .s 

'S'S 

SxrSx";:" 

x.  X  ; 
X:  \'C  X"X  .  X  s-S 
x'-xSX  X  Ssf  ^ 
XX'  'vXs '  X-XxSX 
-X 

sxxc  x'f  r-'X' 


X" 


,  ?  sSi: 

XXXXXiSSiS 


Customer  Support  .7  Future  Requirements  .075  Unit  Performance  .225 

Dynamic  Sensitivity  of  Commanders  Values  Ranking 


Altcoitive 

UclK9 

Absnilive 

\mty 

DU9_13 

0.470Z75 

IW1_3 

3.681797 

ixr9j 

0J37705 

DUIJ 

3.7668Z3 

•■a  s 

IW9_9 

0.893092 

DU6_7 

3.788125 

IXJ9  4 

0549078 

1XJ6_6 

3318200 

D09J 

1213158 

0O5_2 

3326445 

DU9_18 

1221943 

DU5_3 

3579107 

V  sX  5 

DUB  6 

1247820 

DU4J 

4038453 

IXJ6  4 

1248610 

0U3.1 

4.120893 

13U9  19 

1538325 

DU6_2 

4207522 

-  '  ^  s  ^  s 

006  1 

1536940 

D09_12 

4308838 

iS&XS&W-giglSSS 

DU9J5 

1.764185 

s'? 

DUS  1 

4363893 

IXJ9_10 

2J044367 

V? 

DU8_3 

4.608048 

DU9_I4 

1153390 

A  X** 

DU10_2 

4.634527 

>Sss  5  s  s 

DU9_n 

1244285 

DU8_5 

4.714535 

DU9J6 

2301365 

Dm_4 

4390973 

\  N  'Cl:  - 

IX;9  30 

2356520 

s> 

DU4_2 

4394877 

1  sX  s'  s'  !$ 

DU9  17 

1481747 

"CX 

DU4_3 

4.908920 

DU9J7 

2491563 

s 

DU3_2 

4.982950 

DU6J 

1639440 

0U3_5 

4.999507 

DU6  3 

1958627 

xmm 

DU7J 

5.119153 

"  ' 

DU1_2 

3014403 

'  s'  i 

DUIOJ 

5349755 

1XJ9  8 

3022185 

0U4_4 

5386280 

DU1_5 

3025330 

DU4_6 

5399285 

DU9_6 

3075573 

0U2,3 

5553253 

DU3_4 

3.253195 

DU2_2 

5561290 

DU9J 

3323362 

DU2J 

5.741835 

DU9J 

3  3  84920 

DU2_4 

5.743353 

DUS  1 

3496457 

DU8_2 

5.751175 

DU3J 

3532105 

DU3_7 

5.793018 

DUIJ 

3.644502 

DU4J 

5.799285 

IXJ9J1 

3.652200 

DU4_7 

5.981115 
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Appendix  K.  Sensitivity  Analysis 


Customer  Support  .667  Future  Requirements  0  Unit  Performance  .333 
Dynamic  Sensitivity  of  Commandeis  Values  Ranking 


Alternative 

Utility 

AUonalive 

UtiHQ' 

DU9  13 

0.384667 

DU9_I1 

3.400200 

DU9_1 

O.S08S0Q 

DU5_2 

3.432233 

DU9_4 

0.820733 

DU5_3 

3.519683 

DU9_9 

0.891100 

DU1_1 

3.535400 

DU9_2 

1.103517 

DU6_7 

3.558317 

DU6_4 

1.176867 

DU3_1 

3.567167 

DU3_6 

1532050 

DU6_6 

3.590817 

DU9_18 

1564783 

DU4_5 

3.767617 

DU9_19 

1.361200 

DU8_1 

3.793317 

DU6_1 

1.611533 

DU10_2 

3.886350 

issssMSSitii 

DU9  15 

1.662833 

DU6  2 

3.893150 

WS;iW« 

DU9  10 

1.888517 

DU9  12 

3.980183 

S  si'  i^  i"  s 

DU9_14 

1.973733 

Dl»_3 

4.097017 

DU9_21 

2.026700 

DUS _5 

4.102233 

SsS'-^  's' A  S 

DU9J6 

2.130383 

DU8_4 

4597783  5  •s^sss  '  '5:  ^ 

DU9_17 

2524933 

DWjS 

4571100 

S  >'s  i  S  J:  t 

DU9_20 

2581783 

DU3_5 

4586967 

DU9_7 

2565467 

DU7_1 

4506283 

DU6_5 

2J10617 

■•is 

DU4_2 

4.704717 

DU6_3 

2.749883 

DU4_3 

4.743333  V*1  V  S-'-'S.".?  ^ 

DU9_8 

2A46200 

DWJ! 

4.744733  »  •%  S 

DU9_6 

2.869817 

DU2_2 

4.750717 

DUl_2 

2.882217 

s  r 

DU2_1 

4.907300  ■i.  5  «  -5 

DU1_S 

2587267 

DU2_4 

4553067  : 

O  5i. 

DU3  4 

2.902367 

DU10_I 

5.005883  ft.  s 's  '  ^  s  s‘*’*ft 

DU9  5 

3.054850 

<.S  S  JJ. 

DU4_6 

5.103533  i 

DUS_l 

3.103667 

DU3_7 

5.134567  S.'  <: 

DU9J 

3.173933 

S  -  $  -i:: 

DU8^ 

5.135967 

Din_3 

3506817 

DlMjt 

5.232333  ^  ^ 

DU1_6 

3578283 

s  ^  5  ' 

DU4_1 

5.465000  $ 

DU1_3 

3593717  5. 

DU4J7 

5.711100  i 

A"  sl$ 

Customer  Support  .5333 

Future  Requirements  .2 

Unit  Perfonnance  .2667 

Dynamic  Seinsitivity  of  Commands  Values  Ranking 

Altemidve 

UtOi^ 

AlUojitlve 

UtOity 

DU9_1 

0.447280  i 

S' 

DU9_11 

3533200 

DU9_13 

0.484733  i 

th 

DU3_3 

3549613 

-s'  '  V  s^ 

DU9_9 

0.782880 

-• 

DU5_1 

3.467253 

DU9_4 

0.946707  \  i 

DU3_4 

3.498453 

ss  ss^  s 

DU9_2 

1.028053  SVS 

DU5^ 

3.625387 

.•.s.vss!.:*.s:.!.s:.?*,sss 

DU6_4 

1.069093 

DU5_3 

3.746787 

s'  "  J 

DUS  6 

1.166720  ^ss' 

DUejl 

3.774760 

DU9_18 

1.198547 

3.856173 

'  s  M 

DU6_1 

1.407107  V 

■5 

DU9_I2 

3.960467 

DU9  19 

1.442400 

DU3_I 

4548213 

DU9_15 

1519827 

DUB  1 

4562013 

DU9_10 

1.788813  i 

s'  § 

DU4  3 

4.408387 

■s'  .•  \  s'.*  ^ 

DU9_16 

1.952507  s 

Ss^ 

DU4_2 

4.418173 

N*M*:*NS':*».s:*is>N«so 
\  s  ..S  > 

DU9_14 

1.967307 

DU8_3 

4554093 

DU9_20 

1.967787  ^  !$ 

DU4_4 

4.773147 

DU9_2I 

2.072760  $ 

DU8_4 

4.818427 

SVA.SS.VSSS?AiO.V».V.SSSSX» 

DU9_7 

2.145333 

i  ? 

DU4_6 

4.833027  H 

lilisiMMss 

DU6  5 

2592373  •• 

'  ^ 

DU8_5 

4.836627  ' 

1;  •■  -  Ns 

DU9_17 

2532827  s 

X*  ft  i 

DU3_2 

4.869733  ' 

DU1_2 

2536973 

S  s  i 

DUI0_2 

4.989440 

DU1_5 

2.341013  % 

s'  ?  ' 

DUlO.l 

5.041947 

DU9_8 

2.598960  ' 

s  ss  .5 

DU3_5 

5.143320 

'  ' 

DU9_6 

2.603093  N 

"ft  \ 

DU4J 

5578960 

DU6_3 

2.732307 

DU4_7 

5502840  ^ 

DU9  3 

2.798987 

DUe_2 

5564533 

DU9  5 

2.863000 

DU7_1 

5569907 

DUIJ 

3.040560 

DU2_2 

5.718173 

DU1_6 

3.049107 

DU2_3 

5.720107 

DU1_3 

3.099693 

DU2_4 

5.836573 

DU6_7 

3.138133  ' 

DU2_1 

5.845760 

DU6_6 

3.173533 

DU3_7 

5.933413 
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Appendix  K.  Sensitivity  Analysis 


Customer  Support  .685714  Fumre  Requirements  .114286 


Unit  Performance  .2 


Dynamic  Sensitivity  of  Commandeis  Values  Ranking 


Altcmalive 

UtiH^ 

AUanative 

Udli9 

DU9_13 

0.496800 

DU1_3 

3.680949  ^ 

DU9_1 

0.S33131 

DU1_1 

3.729191 

DU9_9 

0.877917 

DU6_7 

3.760929 

DU9„4 

0.985409 

DU6_6 

3.7910  1 

DU9_18 

1.206360 

DU5_2 

3.910354 

DtJ9_2 

1.218040 

DUS_3 

4.077183 

DU3_6 

1.240740 

DU4_S 

4.089794 

DU6_4 

1.243463 

DU6_2 

4235520 

DU9_19 

1.575229 

DU3_1 

4297289 

DU6_1 

1.611366 

Di;9_12 

4.352971 

DU9J5 

1.758234 

DUS  1 

4226646  WM 

DtJ9_10 
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